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ABSTRACT 

This  document  describes  the  results  of  Phase  1  of  a  multi-year 
program  intended  to  identify  and  validate  tools  and  techniques  that  can  be 
used  to  assist  human  factors  engineering  (HFE)  practitioners  during  the 
military's  normal  weapon  system  acquisition  process.  Once  validated,  the 
tools  will  be  supplied  to  U.S.  Navy  laboratories  where  HFE  practitioners 
can  benefit  from  the  assistance  of  these  tools.  A  detailed  literature 
review  was  conducted  to  obtain  data  from  previous  surveys  of  HFE  tools  and 
information  on  how  such  tools  may  be  evaluated  and  validated.  A  HFE  Tool 
Taxonomy  was  developed  to  categorize  the  various  tools.  This  Taxonomy  was 
based  on;  (1)  five  weapon  system  acquisition  phases,  (2)  115  HFE 
procedures  or  tasks  commonly  carried  out  during  these  phases,  and  (3) 
various  attributes  of  the  procedures  and  tools.  Tool  types  recommended  for 
further  evaluation  and  validation  are  described,  and  the  validation  process 
is  described. 
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INTRODUCTION 

HUMAN  FACTORS  ENGINEERING  FOR  MILITARY  SYSTEMS 

The  traditional  methodology  of  Human  Factors  Engineering  (HFE)  is 
used  during  various  aspects  of  the  military  aircraf t/weapon  system 
development  process.  From  concept  definition  of  a  new  system  through 
engineering  development  and  testing,  HFE  techniques  can  help  define  the 
evolving  interface  between  the  human  element  and  the  system.  In  addition, 
Naval  Air  Systems  Command  Instruction  3900.10  (15  January  1986)  mandates 
that  "When  humans  are  components  of  any  system  or  operate  or  maintain 
equipment  covered  by  this  instruction  fall  phases  of  all  system  and 
equipment  acquisitions,  improvements,  production,  and  support!,  principles 
and  practices  of  human  engineering  shall  be  used  to  define  human  functions 
and  the  means  to  accomplish  them." 

In  the  past,  the  use  of  HFE  methods  and  procedures  has  not  always 
been  systematic.  This  is  generally  true  for  both  in-house  Navy  efforts  and 
for  contractor  efforts.  In  addition,  few  of  the  "tools  of  the  trade"  used 
for  HFE  analysis,  design,  and  test  have  been  validated  in  a  laboratory 
setting  or  field  tested.  Some  tools  have  been  developed  because  of  the 
personal  interest  of  the  developer  rather  than  the  appropriateness  of  the 
tool  to  fulfill  a  specific  need. 

Por  purposes  of  this  study,  the  following  definition  has  been  used. 

"A  HFE  tool  is  something  used  in  performing  an  operation,  task,  or 
procedure  necessary  in  the  practice  of  Human  Factors  Engineering.  A  tool 
may  take  the  form  of  a  software  program,  a  paper  and  pencil  technique,  or  a 
documented  procedure  or  method.  It  may  be  government  owned  or  commercially 
available ." 

Categories  of  tools  other  than  software  are  included  so  that  other 
techniques  (e.g.,  paper  and  pencil)  with  a  high  value  to  the  HFE  profession 
are  not  arbitrarily  excluded.  Valuable  tools,  other  than  software,  could 
become  candidates  for  computerization  to  make  them  more  functional. 

In  order  to  be  effective,  HFE  procedures  and  tools  must  be  used 
properly  and  applied  to  an  appropriate  phase  of  the  aircraft  or  weapon 
system  development.  Additionally,  for  HFE  groups  within  the  Navy  to 
communicate  effectively,  a  consensus  should  be  reached  on  the  use  of  tools. 
This  consensus  should  include,  as  a  minimum, 

1.  What  tools  are  to  be  used, 

2.  When  these  tools  are  to  be  used,  and 

3.  How  the  outputs  of  these  tools  are  to  be  used. 

The  Navy  needs  an  evaluation  framework  and  a  standardized  methodology 
both  for  evaluating  existing  tools  and  for  developing  new  tools.  A  proven 
method  is  urgently  required  to  meet  both  present  and  future  needs. 
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PROBLEMS  WITH  HFE  TOOLS 
Limited  Use  of  Tools 


Human  Factors  technology  development  has  traditionally  involved 
creating  new  methodologies  to  model  various  aspects  of  the  human  component 
in  complex  weapon  systems.  The  human  models  primarily  address  cognitive 
processing,  visual  processing,  and  physical  attributes.  These  development 
efforts  have  recently  resulted  in  numerous  software  packages  that  use 
traditional  data  as  inputs  to  an  algorithmic  process. 

There  has  been  a  proliferation  of  software  tools  that  have  undergone 
extensive  development  efforts  but  have  never  emerged  from  the  laboratory. 
Tools  that  have  emerged  and  have  been  put  to  practical  use  have  not  been 
widely  used  by  a  substantial  portion  of  the  practitioners  in  the  field  of 
HFE.  Fleger  and  others  (1987)  surveyed  the  software  tools  available  to  the 
Human  Engineering  industry  during  1986  and  1987.  They  have  reported  that 
the  use  of  automated  aids  is  not  widespread  among  Human  Engineering 
specialists  within  government  and  industry. 

Why  would  someone  working  in  a  highly  dynamic  field  fail  to  take 
advantage  of  an  automated  tool?  In  general,  HFE  tools  have  proven  to  be 

1.  Unknown ,  that  is,  HFE  practitioners  do  not  know  that  the  tools 
exist  or  the  tools  intended  purpose. 

?.  Costly,  due  primarily  to  the  hardware,  software,  and  support 
personnel  required. 

3.  Time  consuming  to  use,  requiring  as  much  time  and  effort  as 
traditional  methods. 

4.  Training  intensive,  requiring  specialized  courses  and  frequent 
use  to  maintain  proficiency. 

5.  Easy  to  Forget ,  requiring  frequent  relearning,  since  HFE 
practitioners  carry  out  each  specific  type  of  procedure  or  task 
(such  as  workload  analysis)  infrequently. 

6.  Unre 1 iable ,  since  the  output  cannot  be  independently  verified. 

7.  Unavailable ,  since  automated  tools  applicable  throughout  the 
Department  of  Defense  (DOD)  sometimes  have  remained  within  the 
province  of  a  single  service  (e.g.,  the  Air  Force). 

Standardization  of  tools  would  theoretically  help  to  control  the  cost 
of  tool  acquisition  and  the  cost  of  training.  As  a  particular  tool  was 
used  over  a  period  of  time,  the  repeated  usage  would  provide  a  means  to 
assess  reliability.  The  amount  of  time  required  to  use  a  tool  would  also 
become  less  of  a  problem  as  the  proficiency  of  the  users  increased.  A 
related  factor,  timing,  is  using  a  tool  at  the  proper  time  in  the 
development  schedule.  With  better  anticipation  of  the  need  (i.e.,  a 
scheduled  time  for  application  of  the  tool),  valuable  lead  time  could  be 


2 


NADC-91 038-60 


gained.  The  time  required  to  use  the  tool  effectively  can  be  spread  out  on 
the  front  end  of  the  process. 

Inappropriate  Use  of  Tools 

The  problem  of  not  using  potentially  valuable  tools  is  compounded 
when  tools  are  used,  but  not  used  appropriately.  Inappropriate  uses  of  HFE 
tools  can  be  placed  in  three  categories. 

1.  Tools  have  been  used  at  inappropriate  times  in  the  weapon  system 
development  process  (i.e.,  too  early  or  too  late  to  affect  design 
decisions ) . 

2.  Tools  have  been  used  at  appropriate  times  but  their  outputs  have 
been  misapplied  when  decisions  were  made  concerning  operator  or 
maintainer  equipment. 

3.  Tools  have  been  used  for  purposes  for  which  they  were  not 
intended,  such  as  for  proving  compliance  with  design  requirements 
"by  analysis"  in  place  of  a  formalized  demonstration  or  test,  or 
for  some  application  for  which  the  tool  was  not  designed. 

Lack  of  Tool  Validation 

Some  tools  are  used  even  though  their  outputs  have  not  been 
validated.  Properly  designed  and  executed  validation  studies  point  out 
deficiencies  if  these  exist.  But  validation  often  is  considered  an 
unacceptable  expense  following  the  cost  of  developing  the  tool.  Lack  of 
validation  continues  to  undermine  the  effective  use  of  tools  by  the  Human 
Engineering  profession. 

ASSUMPTIOMS 

This  investigation  did  not  initially  assume  that  new  tools  must  be 
developed  to  enhance  the  working  climate  of  the  Human  Engineering 
specialist.  Rather,  it  was  assumed  that  tools  with  potential  value  are 
available  now  or  are  currently  under  development  and  in  a  form  suitable  for 
evaluation.  The  potential  value  of  a  tool  will  be  tied  to  Human 
Engineering  tasks  associated  with  the  weapons  system  acquisition  cycle. 

The  assumptions  for  this  project  include  the  following. 

1.  Useful  tools  for  crew  station  analysis,  design,  and  evaluation 
are  needed  to  achieve  effective  crew  station  integration. 

2.  A  variety  of  potentially  useful  crew  station  analysis,  design, 
and  evaluation  tools  currently  exist  or  are  under  development. 

3.  Assessment  of  specific  tool  utility  and  validation  has  been 
minimal ,  so  far . 
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GOAL  AND  OBJECTIVES 

The  goal  of  this  study  is  to  initiate  a  multi-year  program  that  will 
result  in  a  set  of  systematic  procedures  for  ensuring  that  HFE  techniques 
are  properly  applied  to  military  systems,  with  emphasis  on  Naval  aircraft 
crew  stations.  With  the  above  assumptions  in  mind,  this  goal  will  be  met 
by  identifying  and  validating  HFE  tools  which  Navy  Human  Engineering 
practitioners  can  use  for  carrying  out  established  HFE  tasks  and 
procedures.  These  tools  then  will  be  provided  to  the  Navy  Laboratories  for 
use  by  HFE  practitioners  during  the  weapon  system  acquisition  cycle. 

As  noted  above,  what  tools  are  to  be  used,  when  to  use  them,  and  how 
to  use  the  outputs  must  be  specified.  A  tool  val idat ion  framework  and 
standardized  methodology  must  be  provided  through  this  program.  The  end 
result  will  be  tools  that  aid  in  development  of  military  aircraft  and 
weapon  systems--systems  that  can  be  operated  and  maintained  by  military 
personnel  rapidly,  accurately,  safely,  with  ease,  and  with  satisfaction. 

In  order  to  meet  this  goal,  the  overall  program  effort  has  bee'- 
divided  into  two  phases.  Several  objectives  have  been  identified  for  the 
first  phase  (6.3  Advanced  Development  Level). 

1.  Determine  and  document  the  various  kinds  of  tasks  and 
procedures  comraunly  carried  out  by  HFE  practitioners  at  the 
Navy  Laboratories  to  support  crew  station  development. 

2.  Categorize  these  tasks  and  procedures  according  to  the  phase  of 
the  military  weapon  system  acquisition  cycle  in  which  they 
normally  are  applied. 

3.  Identify  and  describe  valid,  existing  HFE  analysis,  design,  and 
evaluation  tools. 

A.  Based  on  the  phases  of  the  military  acquisition  cycle  and  the 
HFE  tasks  and  procedures  carried  out  during  each  phase 
(Objective  2),  develop  an  appropriate  taxonomy  that  can  be  used 
to  categorize  and  to  compare  the  identified  HFE  tools. 

5.  Develop  methodologies  that  can  be  used  first  to  evaluate 
(through  analysis)  the  utility  of  the  identified  tools  and  then 
formally  to  val  idate  (through  testing)  the  best  tool 
candidates,  for  use  in  each  of  the  HFE  task  categories. 

6.  Eva  1 uate  and  compare  the  identified  existing  HFE  tools  and 
select  the  most  promising  for  acquisition  and  formal 

va 1 idat ion . 

7.  Identify  those  significant  HFE  tasks  and  procedures  for  which 
no  promising  tool  currently  exists  or  is  under  development,  and 
determine  whether  a  tool  is  necessary  for  optimum  human 
engineering  of  military  systems. 
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8.  Deterraiae  when,  where,  and  by  whom  formal  validations 

fObjectives  5  and  6)  will  be  carried  out  for  existing  tools 
identified  as  promising  candidates. 

Once  the  first  phase  is  complete,  the  second  phase  (6.4  Full-Scale 
Development  Level)  will  begin.  Objectives  for  this  level  include  the 
f o I  lowing . 

1.  Obtain  the  promising  HFE  tools  identified  in  Objective  6  of 
Phase  I,  and  conduct  formal  val idations  of  them,  using  the 
validation  methodology  developed  in  Objective  5.  Identify  the 
o..e  (or  perhaps  two  or  three)  best  tools  for  each  of  the  HFE 
tasks  or  procedures. 

2.  Develoo  new  HFE  tools  for  those  Human  Engineering  tasks 
identified  in  Obiective  7  of  Phase  T  as  requiring  such  tools 
but  for  which  tools  are  not  now  available. 

1.  T^entify  various  sets  of  va'idated  tools  that  can  be  included 

in  "tool  kits,"  each  kit  containing  those  specific  kinds  of  HFE 
tools  needed  by  Human  Engineering  practitioners  at  each  of  the 
Navy  Laboratories.  "Tool  kit"  components  will  be  based  on  the 
Laboratorv's  mission  and  on  the  kinds  of  HFE  tasks  and 
procedures  carried  out  (as  determined  in  Objective  1,  Phase  I). 

4.  Provide  each  Navy  Laboratory  with  the  appropriate  "tool  kit" 
and  the  training  necessary  to  ensure  that  HFE  personnel  learn 
to  use  the  tools  effectively. 

APPROAC-t 

In  order  to  jnvolve  all  of  the  intended  Laboratory  user  groups  on  the 
front  end  of  this  process,  a  tr  u  1 1  i -Laboratory  team  approach  has  been 
initiated.  Thi®  team  consists  of  Human  Engineering  representatives  from 
the  Naval  Air  Development  Center  (NADC),  the  Naval  Weapons  Center  (NWC), 
and  the  Naval  Air  Test  Center  fNATC).  These  representati ves  will  be 
involved  in  the  review,  evaluation,  and  transition  of  tools  for  crew 
station  analysis,  design,  and  test.  The  team  will  work  towards  devising  a 
common  methodology  to  select  and  evaluate  these  tools,  to  ensure  end  user 
acceptance  of  the  resulting  "tool  kits." 

The  elements  of  the  project  organization  include  the  following. 

1.  Sponsors  -  6.3  -  NAVAIRSYSCOM  ( CDR  S.  Harris) 

6.4  -  NAVAIRSYSCOM  fPMA  202) 

2.  Management  -  Naval  Air  Development  Center 

(Dr.  N.  Warner  Code  60B5) 

3.  Performing  Organizations 

a.  Naval  Air  Dev'^lopment  Center  (Code  6022) 
h.  "aval  Weapons  Center  (Code  316?) 
c.  Naval  Air  Test  Center  (SY-72) 
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This  organization  is  depicted  by  Figure  1, 


Figure  1.  Human  Factors  Engineering  Tool  Evaluation  Project  Organization 


TASKS  AND  MILESTONES 

The  major  tasks  associated  with  the  initial  FY-90  effort  include  the 
fol lowing . 

1.  Literature  review.  Conduct  a  review  of  current  literature  to 
ascertain  how  HFE  tools  have  been  developed,  used,  and  evaluated 
and  to  learn  of  validation  methods  that  may  be  applied  to  this 
effort. 

2.  Taxonomy  development.  Develop  an  organizat iona 1  framework  that 
provides  for  the  initial  assessment  and  selection  of  tools  for 
further  evaluation. 

3.  Validation  process  development.  Whereas  the  tool  taxonomy  will 
serve  for  evaluating  and  selecting  tools,  a  metric  or 
experimental  paradigm  is  needed  to  validate  each  tool  or  class  of 
tool s . 

The  overall  project  schedule  by  phase  is  included  as  Figure  2. 

A  related  effort  includes  performing  informal  evaluations  of  Cockpit 
Automation  Technology  (CAT)  and  Army-NASA  Aircrew/ Aircraft  Integration 
(A3I).  These  evaluations  are  being  conducted  by  NADC  and  NATC  personnel 
respectively. 
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Figure  2.  Human  Factors  Engineering  Tool  Evaluation  -  Tasks  and  Milestones 
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Evaluations  of  these  two  systems  are  being  monitored  to  determine  the 
approach  and  methodology  being  used.  Both  systems  are  in  the  Beta  phase  of 
testing.  Lessons  learned  may  be  available  that  are  related  to  the 
evaluation  process,  the  metric  used,  and  the  specific  evaluation  technique 
for  the  tool.  Technical  Reports  on  these  evaluations  are  expected  during 
FY-90. 


RESULTS  OF  THE  LITERATURE  REVIEW 
PREVIOUS  SURVEYS  OF  HFE  TOOLS 

Two  major  survey-type  efforts  have  attempted  to  link  Human 
Engineering  technologies  and  tools  to  the  Human  Engineering  requirements 
that  are  associated  with  the  weapon  system  acquisition  process.  The  first 
resulted  in  a  report  by  Baker  and  others  (1979),  entitled.  Human  Factors 
Engineering  for  Navy  Weapon  System  Acquisition.  This  report  tied  the  Human 
Engineering  methods,  techniques,  and  tools  that  were  available  or  emerging 
during  the  mid  to  late  1970s  to  the  acquisition  process.  In  addition. 

Human  Engineering  technology  gaps  were  identified  and  recommendations  made 
to  fill  in  these  gaps.  Unfortunately,  the  technology  gaps  reported  in  1979 
have  not  yet  been  adequately  bridged.  Presently,  with  a  more  complex 
acquisition  process  and  reduced  funding  for  front-end  analysis  and  design 
studies,  a  whole  new  list  of  shortfalls  could  be  documented. 

The  most  recent  survey  report  was  by  Fleger  and  others  (1987), 
entitled.  Advanced  Human  Factors  Engineering  Tool  Technologies.  This 
effort  was  sponsored  by  the  Army  and  also  tied  Human  Engineering 
technologies  to  the  milestones  associated  with  the  weapon  system 
acquisition  process.  The  survey  focused  on  advanced,  software-oriented 
tools  rather  than  on  traditional  techniques.  Advanced  tools  include 
prototype  tools  as  well  as  tools  currently  in  use. 

Overall,  it  was  found  that  the  use  of  advanced  tools  by  Human 
Engineering  practitioners  is  not  widespread.  Many  of  the  automated  aids 
are  considered  novel  alternatives  and  have  been  largely  unexplored.  The 
overwhelming  consensus  among  respondents  was  that  computerized  tools  were 
needed  by  the  Human  Engineering  professional .  Future  tool  development 
should  focus  on  the  following  categories. 

1.  Expert  systems 

2.  Human  Factors  data  base  compendiums 

3.  Workload  prediction 

4.  Automated  task  analysis. 

Human  Engineering  tools  that  were  available  for  review  during  this 
industry  survey  were  evaluated  on  the  following  points. 

1.  In  use/in  development 
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2.  Capability  to  augment/replace  traditional  tools 

3.  Ef fectivity/reliabil ity 

4.  Transportability  (hardware  compatibility) 

5.  Versatility  (useful  in  a  variety  of  settings) 

6.  Appropriateness  (when  used). 

A  tool  taxonomy  was  then  created  consisting  of  nine  attributes. 

1.  Advantages 

2.  Disadvantages 

3.  Phase  of  system  design  to  be  used  for  maximum  effectiveness 

4.  Activity  area  (design,  analysis,  test) 

5.  Role  (how  used) 

6.  Type  (kind  of  tool;  what  the  tool  is) 

7.  Class  (area  of  application:  what  it  does) 

8.  Status  (conceptual,  prototype,  or  operational) 


9.  Cost. 

The  tools  recommended  for  procurement  by  the  Army  all  received  the 
following  ratings. 


1.  Currently  operational  and  commercially  available  for  immediate 
implementat ion 

2.  Good  adaptability  and  high  utility 

3.  Low  or  moderate  cost. 

A  total  of  12  tools  met  these  criteria. 

1.  SIMWAM  (Task  Analysis/Workload) 

2.  ZTTA  (Workload/Perforraance/Tracking) 

3.  MicroSAINT  (Workload/Task  Modeling) 

4.  DART  (Workload;  Assembly/Manufacturing  Operations) 

5.  WOSTAS  (Task  A1 location/Workload ) 

6.  HF-ROBOTEX  (Robotics/Expert  System) 
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7.  GRASP  (Robotics/Reach  Analysis) 

8.  CAR  (Reach  Analysis) 

9.  WORG  (Workstation  Arrangement) 

10.  GEOMOD  (Workstation  Design) 

11.  CADAM/ADAM  &  EVE  (Workstation/Reach  Analysis) 

12.  CAPRA  (Maintenance  Analysis). 

These  were  the  tools  recommended  by  Fleger  and  others  (1987)  for  use 
by  the  Army. 

Since  the  Navy  is  primarily  interested  in  tools  specifically  for 
aviation  related  work  or  tools  that  can  be  applied  to  aviation  type 
problems,  the  tools  listed  above  were  reviewed  for  this  quality.  Of  the  12 
tools  listed  above,  the  following  tools  are  considered  to  be  aviation 
related . 

1.  SIMWAM  (Task  Analysis/Workload) 

2.  ZITA  (Workload/Performance/Tracking) 

3.  MicroSAINT  (Workload/Task  Modeling) 

4.  CAR  (Reach  Analysis) 

5.  GEOMOD  (Workstation  Design) 

6.  CADAM/ADAM  &  EVE  (Workstation/Reach  Analysis) 

7.  CAPRA  (Maintenance  Analysis). 

In  summary,  88  tools  were  surveyed  and  documented  and  12  were 
recommended  for  use  by  Fleger  and  others  (1987).  However,  for  purposes  of 
this  report,  only  7  tools  can  be  considered  as  potential  candidates  for  use 
on  Human  Engineering  problems  related  to  military  aviation. 

Based  on  the  Fleger  survey,  advanced  tools  which  run  on  a 
microcomputer  for  use  by  military  research  and  development  laboratories  and 
test  and  evaluation  activities  would  be  most  valuable  to  the  Human 
Engineering  practitioner. 

In  reviewing  the  Fleger  report,  cost  was  a  major  factor  affecting  the 
final  recommendations .  The  criterion  of  low  to  moderate  cost  may  have 
unnecessarily  reduced  the  list  of  recommended  tools.  An  overall  cost 
rating  was  assigned  to  each  tool  based  on  (1)  acquisition  cost,  (2)  setup 
cost,  (3)  training  cost,  and  (4)  host  computer  cost.  Except  for 
acquisition  cost,  this  cost  rating  may  be  somewhat  arbitrary.  A  tool  rated 
as  high  in  cost  was  not  recommended  in  spite  of  technical  merit.  COMBIMAN 
(Computerized  Biomechanical  Man-Model),  GENSAW  (Generic  Systems  Analysis 
Workstation)  and  the  Siegal-Wolf  Model  are  examples  of  tools  falling  into 
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this  high  cost  category.  All  have  been  used  extensively  during  actual 
weapon  system  development  (COMBIMAN  and  GENSAW  primarily  within  the  Air 
Force).  All  have  iw  initial  acquisition  cost.  However,  these  computer 
based  tools  have  a  high  cost  rating  due  to  significant  costs  associated 
with  setup,  training,  and  the  resources  needed  to  keep  them  operational. 

A  key  finding  of  the  Fleger  report  is  that  validation  has  been 
accomplished  for  only  one  of  the  recommended  tools,  even  though  all 
received  high  marks  for  availability,  adaptability,  and  utility.  The  tool 
reported  to  have  been  validated  is  DART  (Data  Analysis  and  Retrieval 
Technique).  DART  is  a  task  model,  workload,  and  timeline  analysis  tool 
which  has  been  used  in  the  manufacturing/assembly  environment  of  commercial 
and  military  aircraft.  The  report  states  that  DART  has  been  validated 
extensively  through  motion  analysis. 

The  results  of  reviewing  the  1979  and  1987  surveys  linking  Human 
Engineering  technologies  to  Human  Engineering  tool  requirements  may  be 
summarized  as  follows. 

1.  Numerous  tools  with  potential  HFE  applicability  have  been 
proposed.  Some  of  these  have  actually  been  developed  and  have 
been  used  by  the  military. 

2.  Criteria  used  by  previous  investigators  to  determine  whether  a 
tool  will  be  recommended  for  government  procurement  may  not  be 
appropriate  for  this  current  study.  Overemphasis  on  cost  as  an 
evaluation  factor  has  eliminated  several  otherwise  highly  rated 
tools  from  consideration. 

3.  Currently  only  one  of  the  tools  recommended  by  other 
investigators  has  been  adequately  validated.  There  have  been 
validation  attempts  on  several  tools  in  the  areas  of  human 
performance  and  physical  accommodation  but  these  tools  have  not 
been  recommended  due  to  other  factors  (e.g.,  poor  usability). 

4.  No  standardized  or  even  accepted  technique  exists  for  validation 
of  the  various  tools  that  might  be  considered  for  use  by  the 
Navy. 

5.  Tools  recommended  in  the  1979  and  1987  reports  almost  exclusively 
are  useful  for  only  three  of  the  Human  Engineering  tasks  carried 
out  routinely:  (1)  task  analysis/  workload  analysis,  (2) 
anthropometric  evaluations,  and  (3)  workstation  design.  Many 
tools  exist  in  these  categories,  while  other  Human  Engineering 
techniques  definitely  aopear  to  be  under-represented. 

TOOL  TAXONOMY  CONSIDERATIONS 

The  literature  review  revealed  certain  issues  that  must  be  considered 
while  constructing  a  taxonomy  of  HFE  tools.  These  include  the  following. 

1.  The  nature  of  the  process  of  system  design 

2.  Relationship  of  tools  to  the  weapon  system  acquisition  process 
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3-  Integration  level  of  tools  intended  to  be  exercised  jointly  in  a 
stepped  or  layered  approach 

4.  Quality  of  the  user  interface 

5.  Potential  for  theories  and  principles,  such  as  those  expressed  in 
a  model,  to  be  considered  as  tools 

6.  Assessment  of  real  costs 

7.  Measurement  of  the  level  to  which  HFE  goals  are  met,  using 
various  tools. 

System  Design  Process 

Rouse  and  Cody  (1989)  have  characterized  the  first  steps  of  system 
design  as  information  seeking,  in  their  assessment  of  the  criteria  that 
design  engineers  use  for  choosing  human  performance  models.  They  describe 
Che  system  design  procedure  as  a  process  of  gathering  and  transforming 
information  into  a  design.  The  information  transformations  occur  on  two 
1 evel s . 

1.  Seeking,  managing,  and  disseminating  information 

2.  Formulating,  associating,  and  evaluating  information  during  the 
transformation  into  a  system  design. 

Human  performance  models,  as  well  as  Che  output  of  other  Human 
Engineering  tools,  are  considered  information  sources.  From  initial 
mission  analysis  and  concept  formulation  through  full-scale  development  and 
deployment,  information  is  sought  for  various  design  issues.  These  include 
establishing  functional  requirements,  creating  and  analyzing  solution 
alternatives,  investigating  solution  details,  and  designing  and  developing 
solutions.  Information  needs  related  to  Human  Engineering  are  represented 
by  things  such  as  task  allocation,  crew  performance,  and  workspace  design. 

Cody  and  Rouse  (1989)  document  the  three  principal  sources  of 
information  that  a  design  team  will  access  during  the  course  of  system 
design. 

1.  Human  judgement  (opinions  of  team  members,  domain  experts,  system 
users) 


Archives  (past  designs,  standard  practices,  scientific  and 
technical  literature) 


3.  Models  (empirical  studies  using  human  subjects  and  analytical 
tools  using  simulations  of  the  human  component). 

One  important  finding  by  Cody  and  Rouse  is  that  the  frequency  of 
access  of  these  information  sources  by  system  designers  occurs  in  the  order 
listed  above  (first  judgement,  then  archives,  then  models).  Frequency  of 
access  is  determined  by  seven  specific  criteria  a  designer  uses  to  evaluate 
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the  value  of  an  information  source.  Furthermore,  these  criteria  seem  to  be 
applied  as  a  series  of  gates  or  filters  in  the  following  sequence. 

1.  Applicability  (can  the  source  produce  information  that  is 
directly  relevant  to  the  problem?) 

2.  Credibility  (is  the  information  from  the  source  perceived  to  be 
val id? ) 

3.  Availability  (is  the  source  commercially  accessible,  in  a 
development  stage  or  proprietary?) 

4.  Cost  (is  the  cost  of  obtaining  and  using  the  source  acceptable?) 

5.  Interpretabi 1 i ty  (are  outputs  from  the  source  directly  usable  or 
easily  transformed  for  the  purpose  at  hand?) 

h.  Learnability  (can  one  become  proficient  in  producing  information 
with  the  source  in  an  acceptable  period  of  time?) 

7.  Usability  (once  mastered,  is  the  source  easy  to  configure  and 
use? )  . 

Using  these  criteria,  models  are  perceived  by  system  designers  to  be 
weak  when  compared  to  the  other  alternatives  of  human  judgement  and 
archives  (e.g.,  past  designs).  This  is  due  to  the  fact  that  system 
designers  can  be  seen  as  consumers  who  are  trading  off  the  expected 
benefits  and  costs  of  seeking  the  information  needed  to  support  design 
decisions.  Tn  this  case,  models  are  perceived  to  compare  poorly  with  other 
sources  in  terms  of  a  cost-benefit  relationship. 

Weapon  System  Acquisition  Process 

In  keeping  with  the  objectives  of  this  current  effort,  a  tool  must 
relate  to  and  support  the  accomplishment  of  a  Human  Engineering  task 
associated  with  the  weapon  system  acquisition  process.  But  how  do  we 
demonstrate  this  connection?  All  Human  Engineering  tools  could  seemingly 
be  linked  in  some  manner  to  the  major  milestones  associated  with  the 
acquisition  cycle  via  the  traditional  Human  Engineering  tasks.  However, 
the  process  of  acquiring  a  new  or  modified  weapon  system,  in  practice,  has 
many  variations.  Furthermore,  the  traditional  Human  Engineering  tasks  are 
often  not  successfully  linked  to  these  variations  in  the  acquisition 
process . 

Some  general  use  tools  are  not  directly  linked  to  Human  Engineering 
tasks  associated  with  the  acquisition  of  a  weapon  system.  These  include 
sfaristical  packages,  novel  uses  of  spreadsheets,  database  management 
programs,  and  hypertext  packages.  Yet  these  tools  should  be  included  in 
the  taxonomy  for  further  review. 

The  number  of  tools  available  to  support  a  particular  Human 
Engineering  task  in  the  weapon  system  acquisition  cycle  may  lead  to  a 
decision  point.  If  there  is  a  large  number  of  tools  available  to  support  a 
specific  task,  it  is  possible  to  be  selective.  If  there  is  a  limited 
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number,  what  is  available  can  be  accepted,  or  a  recommendation  can  be  made 
to  develop  what  is  needed. 

Integrated  Sets  of  Tools 

Recent  developments  have  shown  that  related  HFE  tools  can  be  at  least 
physically  integrated  for  use  at  a  common  workstation.  As  evaluators  and 
users  of  tools  a  question  must  be  posed:  Does  the  integration  of  various 
tools,  versus  stand-alone  tools,  add  to  the  overall  value  or  effectiveness 
of  the  tools? 

Various  levels  of  integration  are  possible. 

1.  Single-user  workstations  with  individual  tools  packaged  together 
and  selected  via  a  common  user  interface  shell  (e.g.,  main  menu). 

2.  Single-user  workstations  and  common  user  interface  with  multiple 
tools  that  can  be  accessed  and  used  interactively  (e.g..  Wherry's 
Human  Engineering  Tools  (Wlierry  1990),  prototype  system 
implemented  with  hypertext-type  software). 

3.  Multiple-user  workstations  used  to  exercise  multiple  tools  in  a 
step-by-step  approach  to  design;  the  output  files  created  via  one 
tool  may  be  used  independently  or  as  inputs  to  another  related 
tool  (e.g..  Cockpit  Automation  Technology  (CAT)  Program,  Air 
Force  Systems  Command). 

User  Interface  Quality 

A  critical  feature  or  characteristic  of  a  tool  is  the  quality  of  the 
user  interface.  Is  the  tool  useable  with  moderate  training?  First 
generation  automated  HFE  tools  have  been  characterized  by  poor  user 
interfaces.  This  factor  has  limited  their  utility  in  meeting  the  real 
needs  of  HFE  practitioners. 

Models  as  Tools 

The  potential  for  using  a  model  as  a  tool  requires  an  in-depth 
assessment.  Considering  the  de^^inition  for  a  HFE  tool  used  in  this  report, 
where  does  a  model  fit  in?  When  does  a  model  reach  a  state  of  utility 
such  that  it  can  be  considered  as  a  tool? 

Models  have  been  used  by  researchers  to  demonstrate  theories  of  human 
performance  and  have  been  used  by  Human  Factors  specialists  to  answer 
questions  of  feasibility  in  system  design.  The  most  promising  human 
performance  models  are  computational  models  that  can  be  or  have  been 
implemented  in  a  computer.  Pew  and  others  (1977)  state  that  models  can 
take  various  forms.  They  can  be  a  verbal /anal yt ic  statement  of  principles 
or  provide  a  closed-form  mathematical  solution. 

If  not  computerized,  a  model  must  have  certain  features  in  order  to 
qualify  as  a  tool.  There  must  be  input,  a  transformation,  and  output.  For 
purposes  of  this  study,  the  output  must  be  useful  For  the  weapon  system 
acquisition  process. 
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Various  authors  (Pew  and  others,  1977)  have  evaluated  the  quality  of 
models  in  terras  of  reliability  and  validity. 

Reliability  means  that  repeated  applications  yield  the  same  or 
substantially  similar  results.  It  is  difficult  to  judge  the  reliability  of 
recent  models  because  they  have  not  had  the  benefit  of  repeated 
appl icat ion . 

Validity  is  the  correlation  of  the  results  of  a  test  with  an  outside 
criterion  or  independent  measure.  This  correlation  provides  an  index  of 
model  validity.  A  validity  test  seeks  to  answer  the  question,  does  the 
modeling  tool  measure  what  it  is  intended  to  measure? 

The  validation  of  software  tools  that  contain  embedded  models  (e.g., 
human  performance  models)  may  raise  several  questions. 

1.  Is  the  model  logically  consistent? 

?.  Is  sufficient  empirical  evidence  available  to  support  the  model? 

3.  Can  we  perform  the  Human  Engineering  task  without  the  model? 

4.  ho  we  validate  the  model  itself  or  accept  the  model  and  validate 
whether  the  software  tool  uses  the  model  correctly? 

5.  Does  a  tool  with  an  embedded  human  performance  model  (e.g.,  the 
Wickens  Multiple  Resource  Model)  add  to  its  value?  What  added 
value  does  the  model  provide?  An  assessment  of  added  value  may 
take  the  form  of  a  subjective  evaluation,  followed  by  a 
simulation  study  and  comparison  of  predicted  and  observed 
resul ts . 

The  problem  of  using  models  of  the  human  component  of  a  complex 
weapon  system  is  being  addressed  on  a  large  scale  by  the  Army-NASA 
Aircrew/ Aircraft  Integration  (A3I)  Program.  This  program  involves  the 
development  of  a  prototype  of  a  Human  Factors  computer-aided  engineering 
(CAE)  facility  for  the  design  of  helicopter  cockpits.  The  A3I  CAE  facility 
is  an  evolving  set  of  tools,  based  on  models  of  human  performance, 
supported  by  a  flexible  integration  framework.  The  objectives  are  as 
fol lows . 

1.  Review  current  models  of  human  performance 

2.  Identify  those  models  that  would  be  most  useful  for  the  CAE 
fac i 1 ity 

3.  Identify  limitations  of  the  models 

t* .  Provide  guidance  for  the  use  of  these  models  in  the  CAE  facility 

5.  Recommend  research  on  models  and  modeling  that  might  overcome 
existing  limitations. 
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Details  of  an  initial  study  of  using  human  performance  models  for  a 
CAE  facility  were  reported  by  Elkind  and  others  (1989).  The  scope  of  this 
study  was  limited  to  the  visual  and  associated  cognitive  functions  required 
of  pilots  in  the  operation  of  advanced  helicopters,  which  often  fly  under 
low-visibility  and  low-altitude  conditions. 

Assessment  of  Costs 


An  assessment  of  the  cost  of  an  HFE  tool  involves  costs  associated 
with  acquisition,  setup,  training,  and  resources  (hardware,  vendor  support, 
etc.).  For  example,  some  tools  have  been  documented  as  having  a  high 
overall  cost  by  Fleger  and  others  (1987)  even  though  there  is  no 
acquisition  cost.  The  costs  associated  with  setup,  training,  and  resources 
can  add  appreciably  to  the  total  cost. 

For  commercial  products,  vendor  support  can  represent  a  substantial 
and  recurring  cost  and  must  be  assessed.  Evaluation  factors  for  vendor 
support  include 

1.  Viability  of  the  company  supporting  the  tool 

2.  Availability  and  quality  of  documentation 

3.  Availability  of  training. 

Measuring  VThether  HFE  Goals  Are  Met 


An  evaluative  process  must  occur  in  order  to  determine  whether  Human 
Engineering  objectives  are  met  during  the  course  of  system  design.  The 
lack  of  metrics  that  can  be  used  for  the  evaluation  of  the  design  adequacy 
of  a  system  is  a  major  deficiency  of  current  Human  Engineering  technology. 

Measures  of  effectiveness  and  preset  criteria  of  system  adequacy  are 
needed.  A  metric  is  required  to  solve  the  problem  of  what  to  measure  and 
how  this  measurement  will  be  made.  Wlierry  (1990)  has  stated  that  when  the 
metric  involves  a  pass/fail  decision,  three  cons idera*" i ons  must  be 
addressed . 

1.  Operationally  defining  a  criterion  variable  that  can  be  measured 

2.  Defining  the  process  by  which  things  will  be  measured  on  that 
variable 

3.  Defining  where  the  line  is  to  be  drawn  between  "acceptable"  and 
"unacceptable" . 

TOOL  EVALUATION  AND  VALIDATION  METHODS 

Based  on  the  literature  review,  methods  that  can  be  used  for 
evaluation  and  validation  may  be  categorized  as  follows. 
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1.  Subjective  analysis.  Several  techniques  can  be  used  and  various 
kinds  of  questions  answered. 

a.  Informal  demonstration  that  a  tool  produces  reasonable 
results  (Pew  and  others,  1977) 

b.  Does  the  tool  perform  as  advertised? 

c.  Are  the  results  that  are  obtained  plausible? 

d.  Is  the  tool  widely  used  or  seldom  used? 

2.  Experimental  studies.  Compare  actual  human  performance  with 
the  predictions  of  the  tool.  This  technique  can  cover  the  range 
from  part-task  lab  studies  to  real-time  manned  simulations. 

3.  Nonexperimental  studies.  Two  procedures  are  commonly  used. 

a.  Compare  tool  predictions  with  estimates  of  various  parameters 
(i.e.,  operator  efficiency)  by  operational  personnel  (Pew  and 
others,  1977). 

b.  Compare  tool  predictions  with  results  (i.e,  empirical 
evidence)  or  information  obtained  from  field  experience  (Pew 
and  others,  1977). 

4.  Systematic  violation  o^  the  assumptions  on  which  the  model  or 
tool  is  based,  using  simulation  runs.  Under  what  conditions  does 
the  model/tool  maintain  validity?  One  approach  to  this  method  is 
to  use  one  tool  to  validate  another  (Chubb  and  others,  1987). 

For  purposes  of  this  study,  evaluation  and  val idation  are  considered 
to  be  two  separate  steps  in  determining  the  usefulness  of  various  HFE 
tools.  Individual  tools  are  eval uated  using  subjective  analysis.  The  goal 
of  these  evaluations  is  to  decide  if  specific  tools  should  be  considered  as 
candidates  for  formal  validation.  During  the  evaluation  process, 
information  about  each  tool  is  reviewed  to  determine  whether  the  following 
criteria  are  met . 

1.  The  tool  is  useful  for  some  task  or  procedure  carried  out  by  DOD 
HFE  practitioners  during  the  weapon  system  acquisition  cycle,  and 
be  consistent  with  existing  DOD  test  and  evaluation  procedures 
and  criteria. 

2.  The  tool  (and  any  related  required  hardware  or  software)  is 
currently  available,  at  "reasonable"  cost,  or  will  be  available 
within  a  "reasonable"  period  of  time. 

3.  The  tool  produces  results  that  are  realistic,  plausible,  in 
useful  form,  and  similar  to  empirical  results  when  these  exist. 

4.  The  tool  appears  to  be  easy  enough  to  learn  and  to  use  that  DOD 
HFE  practitioners  actually  will  use  it. 

5.  The  tool  is  one  of  the  best  for  its  purpose,  when  compared  with 
other  tools  and  techniques  that  may  be  used  to  aid  HFE 
practitioners  with  a  given  task. 
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Following  evaluations,  promising  HFE  tools  will  be  validated 
objectively.  Methods  such  as  experimental  studies,  nonexperiraental 
studies,  and  assumption  violation  procedures  (as  discussed  above)  can  be 
used  for  this  more  formal  process.  Some  recent  tool  validation  studies 
from  the  literature  are  provided  as  examples  of  how  others  have  approached 
evaluation  and  validation. 

Anthropometric  Tool  Evaluation 

The  goal  of  a  study  conducted  by  McConville  and  others  (1989)  was  to 
assess  the  accuracy  and  reliability  of  the  Automatic  Anthropometric  Data 
Measuring  System  (A.\DMS)  in  obtaining  body  size  measurements  of  Navy 
personnel.  The  results  from  AADMS  were  compared  to  results  obtained  from 
four  other  measuring  systems. 

1.  Two  versions  of  the  Integrated  Anthropometric  Device  (lAD) 

2.  Techniques  used  for  the  1964  anthropometric  survey  of  Naval 
avi ator s 

3.  Techniques  used  in  the  1988  Army  anthropometric  survey. 

The  evaluators  reported  that  the  AADMS  provides  a  level  of 
measurement  reliability  as  good  as  or  better  than  any  of  the  other 
techniques  and  devices  tested.  AADMS  permits  the  measurement  of  more 
relevant  cockpit  accommodation  dimensions  than  does  lAD.  With  its 
automatic  measurement  recording  capability,  it  also  can  instantaneously 
edit  and  evaluate  data.  AADMS  provides  a  printout  of  a  subject's  personal 
anthropometric  data  and  any  aircraft  anthropometric  restrictions  for  that 
subject.  However,  the  present  configuration  of  AADMS  needs  improvement, 
since  it  now  requires  skilled  technicians  for  its  maintenance  (hardware  and 
software ) . 

Method  of  Tool  Evaluation:  Quantification  of  the  differences  betwren 
various  measuring  techniques  by  actual  measurement  of  a  series  of 
dimensions  on  a  sample  of  test  subjects. 

Crew  Station  Design  Tool  Evaluation 

Binder  and  others  (1988)  performed  an  assessment  of  the  System  for 
Aiding  Man  Machine  Interaction  Evaluation  (SAMMIE).  SAMMIE  is  a  computer 
model  used  as  a  crew  station  design  tool.  This  study  used  SAMMIE  to 
perform  a  crew  station  accommodation  test.  The  test  involved  human 
subjects  and  a  crew  station  mockup,  to  test  both  MIL-STD-1333  Zone  1  and 
Zone  2  reach.  The  results  were  compared  with  CAR-IV  crew  station 
accommodation  analysis  on  the  same  crew  station.  Evaluators  ported  that 
agreement  rates  between  the  test  subjects  and  the  respective  computer  model 
showed  that  SAMMT*^  compared  favorably  with  CAR-TV. 

Method  0*^  Tool  Evaluation:  Computer  model-generated  reach  data  were 
compared  to  the  results  obtained  experimentally  using  human  subjects  and  a 
crew  station  mickup. 
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Workload  Analysis  Tool  Evaluation 

Chubb  and  others  (1987)  developed  a  model  for  workload  analysis 
called  Saturation  of  Tactical  Aviator  Load  Limits  (STALL).  Model 
validation  involved  developing  SAINT  (Systems  Analysis  of  Integrated 
Networks  of  Tasks)  equivalents  of  the  STALL  model  and  verifying  these 
against  the  STALL  results. 

Method  of  Toox  Evaluation:  SAINT  models  were  used  for  systematic 
violation  the  assumptions  on  which  STALL  is  based.  If  the  assumptions  are 
valid  in  a  particular  application,  this  demonstration  provides  an  indicator 
of  model  robustness. 


DEVELOPMENT  OF  AN  HFE  TOOLS  TAXONOMY 
HFE  PROCEDURES  TAXONOMY 

A  major  goal  of  this  project  is  to  identify  a  set  of  HFE-related 
tools  for  use  by  Navy  laboratories  during  the  system  acquisition  process. 
Each  tool  must  meet  a  recognized  need  if  it  is  to  be  coi  sidered--a  need 
related  to  the  routine  tasks  carried  out  by  Navy  HFE  practitioners  as  they 
define  and  evaluate  the  work  of  contractors. 

Ti5  be  considered  for  purchase,  each  tool  also  must  be  cost  effective 
and,  if  possible,  the  best  of  its  type.  Thus  tools  used  for  similar 
purposes  must  be  compared  to  one  another.  To  compare  various  HFE  tools,  it 
is  necessary  fir«t  to  categorize  them  into  groups  of  tools  that  have 
similar  purposes.  Two  categorization  schemes  have  been  used  for  this 
process , 


1.  The  weapon  system  acquisition  phases  during  which  HFE  tools 
might  be  used . 

2.  The  established  HFE  tasks  and  procedures  for  which  tools  might 
be  used . 

Using  these  two  categorization  schemes,  a  Taxonomy  of  Human  Factors 
Engineering  Procedures  has  been  developed  to  facilitate  sorting  the  tools 
for  comparison  and  evaluation  (Appendix  A).  That  is,  a  complete,  detailed 
list  was  prepared  documenting  essentially  all  of  the  HFE  tasks 
(approximately  115  procedures)  commonly  carried  out  during  all  of  the  five 
phases  of  system  acquisition. 

Contractor  tasks  are  included  in  that  list  as  well  as  tasks  performed 
by  Navy  personnel  to  ensure  that  all  procedures  are  considered.  Several 
attributes  of  each  procedure  are  incl-ided  (as  discussed  below)  and  a  rough, 
preliminary  tally  is  provided  for  the  number  of  HFE  tools  possibly 
available  to  aid  with  each  task.  Microsoft  Windows  3.0  and  the  Microsoft 
Excel  spreadsheet  program  were  used  to  enter  and  manage  the  information  for 
this  HFE  Procedures  taxonomy. 
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A  listing  of  common  HFE  tasks  serves  several  functions.  As  noted,  it 
can  be  used  to  place  tools  designed  for  the  same  task  into  one  category, 
for  comparison.  It  also  helps  put  the  HFE  procedures  themselves  into 
perspective  so  each  can  be  compared  with  the  others.  During  the  various 
acquisition  phases,  which  tasks  are  the  most  important,  and  so  justify  the 
expense  of  tool  acquisition?  Which  procedures  are  the  most  difficult,  and 
thus  truly  require  the  assistance  of  tools? 

Weapon  System  Acquisition  Phases  as  Tools  Categories 

Since  the  HFE  tools  will  be  used  to  aid  Navy  HFE  practitioners  during 
the  weapon  system  acquisition  process,  the  phases  of  that  process  provide 
one  logical  set  of  categories:  tools  used  during  the  same  acquisition 
phase  can  be  grouped  together.  The  weapon  system  acquisition  process 
usually  is  divided  into  five  phases.  Each  phase  includes  various  tasks 
that  must  be  completed  prior  to  a  major  program  milestone. 

1.  Pre-Milestone  0,  the  mission  analysis  phase. 

2.  Pre-Milestone  1,  the  concept  development  or  program  initiation 
phase . 

3.  Pre-Milestone  2,  the  system  demonstration  and  validation  phase. 

4.  Pre-Milestone  3,  full-scale  engineering  development  phase. 

5.  Post-Milestone  3,  the  production  and  deployment  phase. 

HFE  Procedures  and  Tasks  as  Tools  Categories 


The  kinds  of  procedures  and  techniques  that  commonly  are  used  by 
Human  Factors  personnel  provide  another  way  to  separate  HFE  tools  into 
reasonable  categories:  tools  used  for  the  same  general  purpose  can  be 
included  in  one  category.  Appendix  A  includes  a  comprehensive  listing  or 
taxonomy  of  some  115  HFE  procedures  which  can  be  used  to  classify  tools 
according  to  their  functions. 

For  a  given  program,  these  HFE  procedures  often  begin  with  a  mission 
analysis  and  with  planning  how  the  HFE  program  will  be  carried  out.  They 
continue  with  task,  workload,  display  and  control,  manpower,  job,  and  other 
kinds  of  analyses.  The  procedures  culminate  in  tests  and  evaluations  of 
prototype  or  developmental  systems,  as  needed  to  determine  that  HFE  goals 
have  been  met. 

Figures  3,  4,  5,  6,  and  7  show  some  of  the  Human  Factors  tasks 
normally  performed,  as  these  relate  to  each  of  the  system  acquisition 
phases  and  milestones.  Figure  8  illustrates  how  the  HFE  procedures  and  the 
acquisition  phases  relate  to  one  another  and  to  the  placement  of  tools  into 
groups,  for  this  study. 

HFE  TOOLS  TAXONOMY 

As  developed  for  this  study,  the  HFE  Tools  Taxonomy  database  is  built 
on  the  five  weapon  system  acquisition  cycle  phases  and  115  HFE  tasks 
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MISSION  ANALYSIS  PHASE 

-  I  . 


HUMAN  FACTORS  ENGINEERING  PRODUCT 

DEVELOP  ROLE  OF  OPERATOFVMAINTAINER  AS  PART  OF  A 
MISSION  ELEMENT  NEEDS  STATEMENT  (MENS) 

ACTION 

ASSESSMENT  OF  CONFIGURATION  OPTIONS  IN  VARIED 
OPERATIONAL  SCENARIOS 

HUMAN  FACTORS  ENGINEERING  PROCEDURES 
MENS  PREPARATION 

DEFINE  HUMAN  ENGINEERING  ISSUES/RISKS 
REVIEW  SIMILAR  PROJECTS 
COST/BENEFIT  ANALYSIS 
EXPERIMENTAL  STUDIES 
DATA  ANALYSIS 


Milestone 


Figure  3.  Human  Factors  Engineering  Product,  Action,  and  Procedures  Related  to  the 
Mission  Analysis  Phase  of  the  Weapon  System  Acquisition  Cycle,  Prior  to 
Milestone  0. 


r  CONCEPT  DEVELOPMENT  PHASE 


HUMAN  FACTORS  ENGINEERING  PRODUCT 

ALLOCATION  OF  SYSTEM  FUNCTIONS  TO  OPERATORtMAINTAINER 
AS  PART  OF  THE  DECISION  COORDINATING  PAPER 

ACTION 

ASSESS  RELATIVE  COST  EFFECTIVENESS  OF  DESIGN  OPTIONS 

HUMAN  FACTORS  ENGINEERING  PROCEDURES 

CONDUCT/REVIEW  CONCEPT  STUDIES 
PREPARE/REVIEW  SYSTEM  CONCEPT  PAPERS 
PREPARE  DEMONSTRATIONA/ALIDATION  RFP  INPUTS 
(SPECIFICATIONS,  SOW,  CDRL) 

PLAN  PROJECT  MANAGEMENT 
COORDINATE  LAB  SUPPORT  REQUIREMENTS 
EXPERIMENTAL  STUDIES 
DATA  ANALYSIS 


- N 

'  Milestone  1 

V  J 


Figure  4.  Human  Factors  Engineering  Product,  Action,  and  Procedures  Related  to  the 
Concept  Development  Phase  of  the  Weapon  System  Acquisition  Cycle,  Prior 
to  Milestone  1 . 
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DEMONSTRATION  &  VALIDATION  PHASE 


TASK  ANALYSIS  AND  DETERMINATION  OF  HUMAN 
ENGINEERING  REQUIREMENTS 

ACTION 

INFORMATION/PROCESS  FLOW  ANALYSIS;  TASK  ANALYSIS 


EVALUATE  ALTERNATIVE  DESIGN  CONCEPTS 

IDENTIFY  &  INVESTIGATE  RISK  AREAS 

MISSION  ANALYSIS  &  SYSTEM  MODEUNG 

CONDUCT  FUNCTIONAL  ANALYSIS  &  FUNCTION  ALLOCATION 

CONDUCT  TASK  ANALYSIS  &  CRITICAL  TASK  ANALYSIS 

MANPOWER  REQUIREMENTS 

DERIVE  CREW  STATION  ARRANGEMENT,  LAYOUT. 

&  CONTROU'DISPLAY  CONCEPTS 
EVALUATION  OF  DEMONSTRATION  A/ALIDATION  PROPOSALS 
PREPARE  FSED  RFP  INPUTS 
PLAN/CONSTRUCT/EVALUATE  MOCK-UP 
PLAN/CONDUCT/EVALUATE  SIMULATIONS 
PREPARE/REVIEW  HEPP,  HESAR,  CRITICAL  TASK  ANALYSIS, 

HEDAD-O,  HEDAD-M,  HE  TEST  PLAN,  AND  HE  TEST  REPORT 


Figure  5.  Human  Factors  Engineering  Product.  Action,  and  Procedures  Related  to  the 
Demonstration  and  Validation  Phase  of  the  Weapon  System  Acquisition 
Cycle  Prior  to  Milestone  2. 


FULL-SCALE  ENGINEERING  DEVELOPMENT  PHASE 


DESIGN  OF  OPTIMAL  OPERATOFl-MAINTAINER  INTERFACE 


ACTIQN 

DETAIL  DESIGN:  DEVELOPMENTAL  TEST  & 
EVALUATION 


Milestone  3 


CONDUCT  DESIGN  TRADE-OFF  STUDIES 
RESOLVE  CONFUCTING  REQUIREMENTS 
REVIEW  FUNCTION  ALLOCATION 
REVIEW  MANPOWER  ESTIMATES 
TIMELINBNETWORK  ANALYSIS;  WORKLOAD  ANALYSIS 
REVIEW/CONTINUE  CRITICAL  TASK  ANALYSIS 
DESIGN  OPERA TOR/MAINTAINER  INTERFACE 
PLAN/CONSTRUCT/EVALUATE  MOCK-UP 
PLAN/CONDUCT/EVALUATE  SIMUUTIONS 
PREPARE/REVIEW  HEPP,  HESAR,  HEDAD-O.  HEDAD-M. 

HE  TEST  PLAN.  AND  HE  TEST  REPORT 
PUN/CONDUCT  DEVELOPMENTAL  T&E 


Figure  6.  Human  Factors  Engineering  Product,  Action,  and  Procedures  Related  to  the 

Full-Scale  Engineering  Development  Phase  of  the  Weapon  System  Acquisition 
Cycle,  Prior  to  Milestone  3. 


22 


N  ADC-91 038-60 


Figure  8.  Relationship  of  Human  Factors  Engineering  Procedures  and  Weapon  System 
Acquisition  Phases  to  the  Human  Factors  Engineering  Tools  Taxonomy 
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documented  in  the  Taxonomy  of  Human  Factors  Engineering  Procedures 
spreadsheet  (Appendix  A).  Each  tool  is  categorized  according  to  the 
procedure  or  procedures  for  which  it  is  designed,  and  tools  are  grouped 
according  to  one  or  more  of  the  phases  for  which  each  is  appropriate  (some 
procedures  and  tools  apply  to  more  than  one  phase).  The  HFE  Tools  Taxonomy 
has  been  partially  implemented  using  Microsoft  Windows  3-0  and  Precision 
Software's  Superbase  2  database  management  software. 

Figure  9  illustrates  graphically  the  hierarchical  construction  and 
contents  of  the  HFE  Tool  Taxonomy.  Various  important  attributes  of  the  HFE 
procedures  and  of  the  tools  themselves  are  included.  These  attributes  are 
discussed  below. 


Figure  9.  Overall  Design  of  the  HFE  Tool  Taxonomy  Developed  for  this  Study, 
Including  Attributes  of  HFE  Procedures  and  HFE  Tools 


TAXOMOMY  INFORMATION  RELATED  TO  THE  HFE  PROCEDURES  AND  TASKS 

The  system  acquisition  phase  in  which  each  HFE  procedure  is  carried 
out  is  important,  but  is  only  one  of  the  items  that  must  be  considered  in 
taxonomy  development.  It  is  useful  to  specify  several  other  attributes  for 
each  HFE  procedure  in  order  to  determine  who  in  the  weapon  system 
acquisition  cycle  is  most  likely  to  use  the  procedure,  under  what 
cond i t ions  it  will  be  used,  and  where  it  probably  will  be  used.  It  is 
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useful  also  to  identify  existing,  common  resources  (primarily  documents) 
already  provided  by  the  government  to  aid  HFE  practitioners  with  these 
procedures . 

All  of  these  factors  contribute  to  determining  the  priority  level  of 
each  procedure,  and  thus  also  affect  the  priority  which  should  be  assigned 
to  tools  related  to  the  procedure.  The  following  four  attributes  of  the 
procedures  are  included  in  the  HFE  Tool  Taxonomy. 

1.  The  Navy  organizational  level  at  which  each  of  the  various 
procedures  normally  is  carried  out. 

2.  The  kinds  of  resources  (military  documents)  that  already  are 
commonly  available  to  assist  HFE  personnel  with  each  procedure. 

3.  The  size  a  program  must  be  before  it  is  cost  effective  to 
include  a  given  procedure. 

4.  The  kinds  of  systems  (surface,  air,  etc.)  to  which  each  procedure 
applies . 

Level  at  Which  Tasks  Are  Carried  Out 

The  HFE  tasks  and  procedures  carried  out  during  system  acquisition 
can  be  divided  into  four  levels,  based  on  which  organization  or 
organizational  level  usually  has  primary  responsibility  for  their 
completion  (Merriman,  1986). 

1.  Major  milestones  and  events  associated  with  weapon  system 
acquisition  that  drive  all  other  tasks.  These  tasks  generally 
are  the  responsibility  of  the  system  Program  Manager. 

2.  Systems  Command-level  tasks  (NAVAIR,  NAVSEA,  etc.)  which  drive 
contractor  and  laboratory  efforts.  For  Navy  and  Marine  Corps 
crew  stations,  these  tasks  generally  are  the  responsibility  of 
AIR-5313. 

3.  Tasks  usually  (but  not  always)  carried  out  by  the  contractor 
which  is  responsible  for  system  development. 

4.  Tasks  usually  (but  not  always)  carried  out  by  the  Navy's  support 
Laboratories . 


Existing  Task  Resources 

HFE  practitioners  routinely  make  use  of  numerous  military  standards 
and  specifications  documents  while  carrying  out  their  tasks.  The  following 
documents  are  primary  resources  for  system  acquisition,  providing 
information  about  optimum  systems  designs  to  meet  the  needs  of  human 
operators  and  maintainers,  and  about  how  various  HFE  procedures  (including 
preparation  of  reports)  should  be  carried  out. 

1.  MlL-STO-1472 ,  Human  Engineering  Design  Criteria  for  Military 
Systems,  Equipment  and  Facilities 
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2.  MIL-H-46855,  Human  Engineering  Requirements  for  Military  Systems, 
Equipment  and  Facilities 

3.  Data  Item  Descriptions  (DIDs): 


DI-HFAC-80740, 

Human 

Engineering 

Program  Plan 

DI -HFAC-80741 , 

Human 

Engineering 

Progress  Report 

DI-HFAC-80742 , 

Human 

Engineering 

Dynamic  Simulation  Plan 

DI-HFAC-80743, 

Human 

Engineering 

Test  Plan 

DI-HFAC-80744, 

Human 

Eng ineering 

Test  Report 

DI-HFAC-80745, 

Human 

Engineering 

System  Analysis  Report 

DI-HFAC-80746, 

Human 

Engineering 

Design  Approach  Document- 

Operator 

DI-HFAC-80747, 

Human 

Engineering 

Design  Approach  Document- 

Maintainer 

DI-H-7055,  Critical  Task  Analysis  Report. 

Program  Size  Effects  on  HFE  Procedure  Useage 

Not  all  HFE  procedures  are  necessary  for  each  and  every  system 
obtained  by  the  Navy.  The  amount  of  time  and  money  spent  on  Human 
Engineering  tasks  is  a  function  of  the  size  of  the  project  program.  For 
convenience,  programs  can  be  divided  into  three  size  categories. 

1.  Large ;  a  major  program,  such  as  development  of  a  new  aircraft  or 
man-in-the-loop  missile.  Only  very  large  projects  will  warrant 
the  inclusion  of  most  of  the  tasks  included  in  the  taxonomy. 

2.  Med i um :  a  medium-size  program,  such  as  a  maior  aircraft 
modification. 

3.  Small:  minor  projects  such  as  an  aircraft  modification  or  the 
addition  of  a  new  mode  to  a  missile. 

Kinds  of  Systems  the  Procedures  Are  Used  With 


Some  HFE  procedures  are  specific  to  certain  kinds  of  military 
systems.  The  need  to  analyze  the  operator's  visual  field,  for  example, 
normally  is  unique  to  development  of  aircraft  systems.  Dynamic  simulation 
of  the  operator's  mission  also  usually  is  limited  to  aircraft  systems, 
since  this  procedure  is  very  expensive.  Thus,  HFE  procedures  can  be 
identified  as  useful  for  all  kinds  of  systems,  or  as  being  primarily  useful 
for  shipboard,  subsurface,  aircraft,  ground  vehicle,  or  weapon  systems. 
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TAXONOMY  INFORMATION  RELATED  TO  THE  INDIVIDUAL  HFE  TOOLS 

For  a  tool  to  be  considered  for  inclusion  in  the  HFE  tools  taxonomy, 
it  must  be  related  to  the  system  acquisition  cycle  and  must  be  useful  for 
at  least  one  of  the  HFE  procedures  enumerated  in  the  taxonomy.  Thus,  in 
order  to  specify  how  its  output  will  be  used,  each  tool  is  assigned  to  the 
HFE  procedure  for  which  it  is  most  useful  (resulting  in  numerous  tool 
listings  for  some  procedures,  and  none  for  other  procedures). 

Information  is  needed  in  the  taxonomy  to  evaluate  and  to  compare 
related  systems,  in  order  to  determine  what  tools  are  to  be  used  for 
various  procedures,  that  is,  to  select  the  best  tool  for  a  given  function. 
For  each  tool,  this  information  includes  the  type  of  computer  assets 
required  (if  any),  inputs  required,  outputs  provided,  costs,  etc.  A  total 
of  12  useful  tool  attributes  were  identified  and  are  included  in  the 
taxonomy.  These  tool  characteristics  are  discussed  below. 

Basic  Type  of  Tool 


For  purposes  of  this  study,  tools  can  be  divided  into  two  basic 

types . 

1.  Computer-based  programs,  software,  databases,  etc. 

2.  "By-hand"  or  manual  techniques  that  do  not  require  a  computer  for 

use . 

Computer  Assets  Required 


For  those  HFE  tools  that  do  require  a  computer,  the  type  of  computer 
system  that  is  required  is  very  important.  For  practical  purposes,  the  HFE 
"tool  kits"  that  will  be  developed  must  be  operable  on  reasonably 
inexpensive,  widely  used  systems,  such  as  the  Macintosh  or  an  IBM-PC 
compa  t ibl e . 

Some  tools  may  require  a  specific  computer  operating  system  such  as 
UNIX  (or  even  a  specific  version)  for  computer  compatibility.  Some  tools 
are  built  on  existing  commercial  computer  programs  (such  as  Turbo  Pascal) 
or  packages  (such  as  Lotus  1-2-3).  Users  must  purchase  these  products  in 
order  to  use  the  tool.  When  limitations  and  restrictions  exist,  they  must 
be  clearly  noted. 

General  Class  of  Tool 


HFE  tools  can  be  placed  into  classes  depending  on  the  general  types 
of  products  they  represent.  Approximately  10  classes  were  identified  for 
this  study,  and  are  described  in  detail  in  Appendix  B.  Classes  include  the 
fol lowing . 


1.  Standards,  specifications,  or  sets  of  HFE  principles 

2.  Handbooks  or  textbooks 

3.  Databases  of  HFE-related  data 
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4.  Inputs  to  the  HFE  sections  of  Requests  for  Proposals  (RFPs) 

5.  Crewstation  layout  tools 

6.  Man  modeling  or  anthropometries  tools 

7.  Analytical  techniques  (task  analysis,  etc.) 

8.  Assessment  tools  (workload,  etc.) 

9.  Hardware  design  tools 

10.  Interface  (usually  software)  design  tools 

11.  Simulation  software 

12.  Prototyping  tools 

13.  Engineering  tools 

14.  Manufacturing  tools 

15.  Tools  for  use  during  the  formal  test  evaluation  process 
Inputs  Required  by  the  Tool 

Every  HFE  tool  requires  data  or  information  inputs  in  order  to 
produce  results.  Inputs  usually  provide  data  about  the  system  that  is  being 
designed  or  evaluated  and  about  the  intended  human  operators  and 
raaintainers  of  the  system.  It  is  critical  that  all  required  inputs  be 
readily  available  and  easy  to  enter  into  the  system  (whether  using  a 
computer  or  "by-hand"  technique)  or  HFE  practitioners  will  not  use  the 
tool  . 

Outputs  Provided  by  the  Tool 

All  HFE  tools  also  provide  outputs,  the  end  result  products  after  the 
HFE  task  or  procedure  is  completed.  Obviously  the  outputs  must  be  those 
desired  and  needed  by  HFE  practitioners.  Outputs  also  must  be  in  a  readily 
useable  form,  not  requiring  transformations  or  other  manipulations. 

Integration  Level  of  Tools 


Some  existing  HFE  tools  have  been  designed  for  a  single  purpose  or  to 
meer  a  single  need.  Other  tools  have  been  designed  as  part  of  a  package 
intended  to  meet  several  needs  of  HFE  personnel.  Outputs  from  the  use  of 
one  such  "integrated"  tool  often  can  be  directly  used  as  inputs  to  another 
procedure  in  the  same  package.  All  tools  in  a  given  package  may  be 
necessary  for  the  system  to  work  properly,  but  not  all  of  the  tools  may  be 
of  equal  quality.  Thus,  for  the  taxonomy  each  HFE  tool  is  identified  as 
belonging  to  one  of  two  categories. 

1.  Stand-alone  tools  that  assist  with  one  procedure. 
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2.  Integrated  packages  of  tools  which  help  with  several  (usually  related) 
procedures . 

Tool  Costs 


Some  HFE  tools  (books  and  "by-hand"  techniques)  are  relatively 
inexpensive  and  thus  could  be  included  in  each  "tool  kit"  at  minimal 
initial  cost.  If  computer-based  software  runs  on  a  Macintosh  or  an  IBM-PC 
system,  it  probably  can  be  assumed  that  computer  hardware  already  is 
available  at  the  Laboratories  and  will  not  have  to  be  purchased- -making 
these  systems  also  relatively  inexpensive.  On  the  other  hand,  simulation 
software,  and  prototyping  and  engineering  tools  generally  are  very 
expensive  if  special  hardware  is  required. 

Initial  purchase  is  only  one  part  of  total  cost;  the  costs  of  system 
setup  and  continued  support  and  maintainance  can  be  significant. 

In  general,  tool  costs  can  be  categorized  as  follows. 

1.  Inexpensive  and  could  be  made  available  to  all  the  Navy 
Laboratories:  total  cost  under  $2,000  for  one  package. 

2.  Moderately  expensive  but  could  be  made  available  to  one  or  more 
Laboratories  if  the  product  is  very  good:  total  cost  between 
$2,000  and  $10,000  for  one  package. 

3.  Ex pensive  but  could  be  considered  for  one  or  two  Laboratories  if 
the  product  is  very  good  and  aids  with  several  important, 
difficult  tasks:  total  cost  between  $10,000  and  $25»000  for  one 
system. 

4.  Very  expensive  and  probably  practical  only  for  one  Navy 
Laboratory,  highly  involved  in  that  HFE  procedure:  over  $25,000 
for  one  system. 

Training  Required  for  Tool  Use 


The  amount  of  training  needed  to  use  an  HFE  tool  effectively  is  a 
major  component  of  the  overall  cost  of  using  it--and  a  major  determiner  of 
whether  HFE  personnel  actually  will  use  a  tool.  Training  requirements  can 
be  categorized  as  follows. 

1.  Minimal :  less  than  8  hours  of  training  required. 

2.  Moderate :  between  8  and  40  hours  of  training  required. 

3.  Considerable :  more  than  40  hours  of  training  required. 
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Acquisition  Source  for  Tool 

Existing  HFE  tools  may  be  obtained  from  several  kinds  of  sources. 

1.  Government  Laboratories  and  other  government  facilities  which  have 
developed  the  tool  (or  paid  for  its  development)  so  can  provide  it  at 
cost  to  Navy  Laboratories. 

2.  Universities  and  individuals  who  have  developed  HFE  tool  systems  and 
have  permitted  these  to  enter  the  public  domain,  free  of  cost  to  users 
(and  also  unsupported  by  developers). 

3.  Commercial  system  developers  who  will  sell  (and  usually  support)  their 
HFE  tools. 

Level  of  Product  Support 

The  level  of  product  support  available  from  the  HFE  tool's  vendor,  or 

other  supplier,  including  documentation,  training  assistance,  system 

debugging,  and  program  updates,  can  be  categorized  as  follows. 

1.  High ;  adequate  support. 

2.  Moderate ;  some  support;  problems  may  arise  over  time. 

3.  Low;  minimal  vendor  support  can  be  expected. 

References  Providing  Tool  Information 


Personal  contacts,  or  reports  or  other  documents  describing  each 
promising  HFE  tool,  are  necessary  in  order  to  obtain  additional  information 
needed  for  subjective  evaluations. 

Validation  Procedures  Appropriate  for  the  Tool 

Once  selected  for  further  consideration  via  the  evaluation  process, 
HFE  tools  that  are  used  for  different  purposes  will  require  different 
validation  techniques.  Validation  techniques  can  be  placed  in  several 
categories,  as  discussed  earlier. 

1.  Subjective  analysis 

2.  Experimental  comparisons 

3.  Non-exper imental  comparisons 

4.  Systematic  violation  of  the  assumptions  on  which  this  tool  is  based. 
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EVALUATION  AND  VALIDATION  OF  TOOLS 

The  evaluation  and  validation  of  the  following  tool  has  been  planned 
as  part  of  the  6.4  phase  of  the  Advanced  Technology  Crew  Station  (ATCS) 
Program. 

Anthropometric  Computer-Aided  Design  System  (ACADS) 

One  of  the  first  tools  to  be  evaluated  under  this  program  will  be  the 
Anthropometric  Computer  Aided  Design  System  (ACADS).  Information  about 
this  system  has  been  reported  by  Dunn  (1989).  The  ACADS  procedures  are 
common  with  the  evaluation  procedures  used  in  mock-ups  and  prototype 
cockpits.  This  creates  a  consistent  evaluation  method  for  use  in  research, 
development,  and  test  and  evaluation. 

The  object  of  ACADS  is  to  create  three-dimensional  (,3-D)  computer 
records  of  actual  human  subjects  from  video  recordings.  By  video  recording 
the  human  subject  in  a  particular  ejection  seat  or  other  crew  station  seat, 
the  physical  position  of  the  subject  and  the  spatial  location  of  specific 
parts  of  the  subject's  body  may  be  accurately  determined.  This  video 
record  is  collected  under  varying  conditions  of  movement,  attire,  gravity 
forces,  and  restraint.  The  ACADS -generated  3-D  computer  pictorial  template 
can  be  manipulated  in  any  direction  and  about  any  axis. 

The  pictorial  templates  can  interface  with  computer-  aided  design 
(CAD)  models  of  aircraft  crew  stations.  This  enables  a  designer  or 
evaluator  to  place  the  computerized  3-D  template  of  an  actual  human  subject 
into  the  CAD  crew  station,  adjust  the  subject's  position,  and  accurately 
assess  anthropometric  fit  and  function,  including  reach  and  visual  field  of 
view. 


According  to  Dunn,  this  new  capability  will  aid  weapon  system 
developers  in  evaluating  the  anthropometric  aspects  of  their  crew  stations 
while  in  the  design  and  development  phases.  It  will  also  provide  the 
Government  with  a  valid  evaluation  tool  to  assess  compliance  with  new 
disproportionate  anthropometric  matrix  requirements  which  are  now  specified 
by  the  Navy  and  Air  Force  for  aircraft  weapon  systems.  Accommodation  is 
required  for  a  range  of  crew  personnel,  based  on  a  "disproportionate 
matrix"  of  body  dimensions.  Assessment  of  resulting  cockpit  geometry  while 
in  the  design  and  development  stages  is  a  continuing  problem. 

The  evaluation  of  fit,  function,  and  escape  of  the  aircrew  has 
previously  relied  on  the  use  of  anthropometric  models.  Although  many  of 
these  models  exist,  no  validated  anthropometric  system  is  available  that 
can  evaluate  with  sufficient  accuracy  crew  station  geometry  for  crew 
accommodation  in  a  new  weapon  system.  The  failure  of  models  in  cockpit 
dimensional  evaluation  results  in  part  from  the  inability  to  integrate  the 
human  within  the  crew  station,  due  to  the  differential  effects  of  clothing, 
seats,  restraint  systems,  and  personnel  gear,  as  well  as  anthropometric 
variations.  The  use  of  calculated  rather  than  empirical  measurements  in 
weapon  system  design  is  a  source  of  design  errors  which  have  proven  very 
costly  in  terms  of  restricted  manpower,  restricted  performance  and 
engineering  changes.  This  tool  will  be  validated  by  comparison  with  data 
derived  by  empirical  methods. 
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TOOL  TYPES  AND  TOOLS  CONSIDERED  FOR  EVALUATION 

Based  on  the  work  completed  during  FY-90,  certain  types  of  tools  have 
been  identified  for  further  evaluation.  These  tool  types  primarily  support 
the  front-end  analysis  and  planning  that  has  been  lacking  during  weapon 
system  design  evolutions.  If  evaluations  indicate  that  the  tools  meet 
project  requirements,  validations  may  follow. 

HFE  Acquisition  Support 

DOD  HFE  practitioners  immediate  needs  within  this  tool  category 
include  assistance  with  program  planning  and  the  preparation  of  acquisition 
documents.  Tasks  include  requirements  tailoring  (MIL-H-46855 ) ,  modifying 
data  item  descriptions  (OIDs),  preparing  the  statement  of  work  (SOW),  and 
the  development  and  preparation  of  design  criteria  using  documents  such  as 
MIL-STD-1472D  and  related  specifications  and  standards.  Candidate  tools 
are  listed  below. 

Human  Factors  Engineering  Planning  Aid  (Merriman,  1987)  is  a  tool 
that  applied  off-the-shelf  program  management  software  to  HFE  planning  in 
the  context  of  complex  weapon  svstem  developments.  Interdependencies  are 
created  between  models  of  the  military  system  acquisition  process  and  a  HFE 
program  plan  so  that  changes  made  in  the  acquisition  schedule  causes  the 
HFE  plan  to  be  automatically  tailored.  This  planning  aid  also  supports  the 
definition  of  alternative  HFE  plans  in  anticipation  of  possible  funding 
constraints,  schedule  changes,  or  other  contingencies.  This  software  is 
currently  available  at  the  NWC  and  the  NADC.  No  additional  hardware  must 
be  procured  prior  to  an  evaluation. 

Smart  Contract  Preparation  Expediter  (SCOPE)  is  an  expert  system  with 
embedded  knowledge  of  HFE  considerations  and  military  equipment  design.  It 
is  being  developed  by  the  U.S.  Army's  Human  Engineering  Laboratory  (HEL), 
Aberdeen  Proving  Grounds,  MD.  SCOPE  is  intended  for  accurate  generation  of 
system  acquisition  documentation,  including  a  tailored  MIL-H-46855, 
statement  of  work,  and  data  requirements.  The  current  prototype  version  of 
SCOPE  has  been  tailored  specifically  for  the  U.S.  Army's  Missile  Command, 
HEL  Detachment  at  Redstone  Arsenal,  Huntsville,  AL.  However  a  Navy 
evaluation  would  focus  on  features  of  this  tool  that  could  be  adapted  for 
Navy  use.  This  software  is  available  in  a  prototype  version  with  no 
acquisition  cost.  Additional  hardware  must  be  procured  prior  to  an 
eval uat ion . 

MIL-STD-1472D,  Human  Engineering  Criteria  for  Military  Systems, 
Equipment  and  Facilities  (Hypertext  format)  is  an  efficient  way  of 
manipulating  HFE  design  criteria  for  various  purposes.  The  current  version 
of  this  military  standard  has  been  prepared  in  a  hypertext  stack  format 
which  enables  quick  location  and  extraction  of  specific  items  of 
information.  An  important  use  is  for  the  preparation  of  acquisition 
documents.  Extraction  of  design  criteria  for  application  to  higher  level 
specifications  (e.g.,  a  general  specification)  as  tier  0  requirements  is 
becoming  a  standard  approach  to  acquisition  streamlining.  This  product  is 
now  available  from  the  Crew  System  Ergonomics  Information  Analysis  Center 
(CSERIAC)  for  a  cost  recover^'  fee  of  $75.00.  Additional  hardware  must  be 
procured  prior  to  an  evaluation. 
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System  Analysis 

The  comparison  of  a  baseline  system  with  several  versions  of  an 
upgraded  system  is  an  activity  that  frequently  occurs  prior  to  major  system 
acquisition  decisions.  As  a  participant  in  this  type  of  activity,  DOD  HFE 
practicioners  can  perform  a  comparability  analysis.  The  impact  on  the 
human  component  of  a  system  is  assessed  in  this  type  of  analysis.  One  tool 
has  been  identified  for  evaluation. 

Crew  Requirements  Definition  System  (CRDS)  is  a  methodology  designed 
to  evaluate  the  effects  on  the  performance  of  designated  missions  of 
varying  the  crew  size,  task  start  times,  task  sequencing,  and  task 
allocations  to  crewmembers  or  equipment  (Christ  and  others,  1989).  The 
CRDS  has  been  developed  by  the  Army  Research  Institute  (ARI)  and  will  be 
available  during  FY-91.  A  tool  of  this  type  could  be  used  to  support  an 
internal  Navy  analysis  of  system  concepts  and  options  or  to  evaluate  the 
results  of  an  analysis  by  a  contractor.  This  software  is  available  from 
Aid  with  nc  acquisition  cost.  No  additional  hardware  must  be  procured 
prior  to  an  evaluation. 

Crew  Acc  omodation  Analysis/Reach  Assessment 

The  need  for  a  method  of  predicting  crew  accommodation  and  assessing 
reach  acceptability  that  can  be  used  early  in  a  design  process  is  well 
accepted.  Currently  no  tool  meets  these  needs  adequately.  One  candidate 
for  evaluation  has  been  identified. 

Crewstation  Assessment  of  Reach  (CAR)  is  a  design  tool  for  evaluating 
population  accommodation  in  a  workspace  and  the  reachability  of  the 
hardware  components  that  make  up  a  workspace  (Harris,  R.  and  lavecchia,  H., 
1984).  Numerous  evaluations  of  CAR  have  been  completed  by  interested  user 
groups.  Due  to  prior  attempts  at  validation,  the  limitations  oF  the  model 
are  known  and  well  documented.  If  improvements  for  the  CAR  model  are 
desired,  modifications  to  correct  the  known  deficiencies  can  be  undertaken 
in  a  cost-effective  manner.  Also,  guidance  on  appropriate  applications 
(i.e.,  when  to  use)  can  be  added  as  supplemer.tary  information  to  CAR  user 
documentation . 

A  proposed  "standard"  application  of  CAR  is  for  the  evaluation  of 
conceptual  or  preliminary  workstation  designs  prior  to  mock-up 
construction.  Several  alternative  designs  can  be  evaluated  quickly  and  the 
output  cat  be  used  as  an  input  to  preliminary  drawings  and  mock-up  design. 

Currently,  CAR  has  many  government  and  commercial  users.  As  a 
modeling  tool,  it  remains  inexpensive,  easy  to  use,  transportable  (i.e., 

IBM  PC  compatible)  and  can  provide  a  quick  turn-around  of  recommendations 
related  to  an  important  aspect  of  physical  workstation  design.  This 
software  is  currently  available  and  in  use.  No  additional  hardware  must  be 
procured  prior  to  an  evaluation. 

Rapid  Prototyping  of  Hardware  and  Software 

The  primary  issue  with  this  class  of  tools  centers  around  instituting 
"standard"  tools  that  can  be  used  by  both  Navy  activities  and  innustry- 
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with  common  tools,  the  results  of  rapid  prototyping  can  be  more  efficiently 
evaluated  and  transitioned  to  detail  design.  Standardized  tools  are  needed 
for  addressing  hardware  design  issues  (e.g.,  crew  station  layout  and 
arrangement)  as  well  as  software  design  issues  (e.g.,  display  formats  and 
logical  flow  of  activities).  Candidate  tools  are  listed  below. 

Apple  HyperCard  is  a  hypertext  (i.e.,  nonlinear  text)  software 
product  that  is  useful  for  linking  or  cross  referencing  between  related 
text,  graphical,  or  combined  text /graphical  units.  Software  products 
falling  into  the  hypertext  family  exhibit  a  wide  range  of  capability  in 
handling  graphical  information.  The  ability  to  link  non-textual  units 
(e.g.,  diagrams)  has  been  the  prime  driver  behind  the  use  of  HyperCard 
(Apple  Computer,  Inc.,  1988)  as  a  rapid  prototyping  tool.  Novel 
applications  of  this  product  are  prevalent  within  the  HFE  community.  This 
is  a  standard  software  product  that  is  currently  available  and  in  use  at 
the  NADC  and  the  NATC.  Additional  hardware  must  be  procured  prior  to  an 
evaluation. 

ProtoTymer  (Miller,  D.  and  Stone,  A.,  1989)  is  a  software 
instrumentation  package  that  can  record  human  interaction  (e.g.,  button 
hits)  with  a  HyperCard  (Apple  Computer,  Inc.,  1988)  application.  All  of 
the  events  of  a  user  interface  session  can  be  recorded  for  further 
analysis.  The  information  can  be  used  to  assess  learning  time,  number  of 
errors,  and  other  performance  indices.  This  is  a  standard  software  product 
that  is  currently  available.  Additional  hardware  must  be  procured  prior  to 
an  evaluation. 

VAPS  (Virtual  Prototypes,  Inc.,  1988)  is  a  software  tool  that 
addresses  three  problems  that  are  common  to  all  control  and  display 
systems:  display  formats,  operator  interface  logic  and  human  interface 
with  other  systems.  The  VAPS  can  support  the  development  of  solutions  to 
these  design  problems  simultaneously  through  fully  dynamic  system 
evaluations.  This  software  runs  on  Silicon  Graphics  workstations.  This 
system  is  currently  available  and  in  use  at  both  the  NADC  and  the  NATC.  No 
additional  hardware  must  be  procured  prior  to  an  evaluation. 

Hardware  Specification  and  Design 

LIGHT/CRT  MODEL  is  an  engineering  and  human  factors  analysis  tool 
that  is  currently  under  development  at  the  NADC  (Penn,  1990).  This  model 
will  be  used  to  assess  the  impact  of  ambient  light  conditions  on  CRT 
luminance,  shades  of  gray,  and  color  contrast  prior  to  the  availability  of 
actual  hardware.  Based  on  inputs  that  include  actual  or  desired  hardware 
characteristics  and  illuminance  conditions,  the  imagery  generated  by  the 
model  will  simulate  the  interaction  between  the  CRT  performance  and  the 
environment.  A  preliminary  version  of  the  software  will  be  available 
during  FY-91.  Additional  hardware  must  be  procured  prior  to  an  evaluation. 

VALIDATION  PROCESS 

Prior  to  any  validation  study,  it  must  be  determined  that  the  tool 
and  its  output  are  realistic  for  the  HFE  procedure  for  which  it  is 
intended.  The  type  of  level  of  validity  to  be  determined  should  be  related 
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to  the  priority  of  the  tool  within  the  scope  of  the  HFE  tasks  discussed  in 
this  report. 

Thus  the  first  step  in  the  validation  process  will  involve  reaching 
agreement  on  what  tools  are  to  be  selected  for  validation.  This  group  of 
tools  for  validation  can  be  determined  by  evaluating  various  candidates. 
This  evaluation  phase  will  be  helpful  to  the  decision  process. 

Once  the  tools  to  be  validated  are  known,  the  level  of  validation  to 
be  pursued  can  be  set  and  an  appropriate  method  for  validation  can  be 
devised.  The  types  of  validity  to  be  considered  for  various  tools  include 
the  following. 

Content  validity  is  how  well  the  tool  covers  the  domain  that  it  was 
designed  to  cover.  The  tool  is  reviewed  for  completeness.  This  type  of 
validity  seeks  to  determine  if  the  tool  was  constructed  adequately  by 
reviewing  the  rational  basis  for  the  tool  and  the  underlying  principles 
that  went  into  its  construction.  The  evidence  for  content  validity  is 
logical  rather  than  statistical. 

Concurrent  validity  is  based  on  how  well  the  outputs  of  the  tool 
under  evaluation  corresponds  to  outputs  or  predictions  of  other  tools 
designed  to  address  the  same  domain.  Concurrent  validity  is  a  type  of 
criterion-related  validity.  Studies  of  criterion  validity  assess  the 
simultaneous  relationship  between  the  tool  and  a  criterion. 

Predictive  validity  is  how  well  the  tool  serves  a  predictive  purpose. 
This  type  of  validity  is  used  for  the  purpose  of  forecasting.  This  is  an 
important  type  of  validation  for  a  modeling  tool.  Predictive  validity  is 
also  a  form  of  criterion-related  validity.  The  purpose  of  a  tool  may  be  to 
predict  the  likelihood  of  succeeding  on  the  criterion. 

Construct  validity  is  a  necessary  step  when  the  content  of  the  domain 
of  interest  is  not  clearly  defined  or  when  a  criterion  is  not  available 
which  is  accepted  as  adequate  for  the  domain  of  interest.  This  type  of 
validity  is  established  through  a  series  of  activities  in  which  a 
hypothetical  construct  is  defined  and  a  tool  is  simultaneously  developed  to 
measure  it.  Construct  validation  is  an  on-going  process  that  involves 
assembling  evidence  to  support  the  validity  of  the  output  generated  by  a 
tool . 
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SUMMARY 

The  goal  of  the  study  reported  in  this  document  is  to  initiate  a 
multi-year  program  that  will  result  in  a  set  of  systematic  procedures  for 
ensuring  that  HFE  techniques  are  properly  applied  to  military  systems  (with 
emphasis  on  aircraft  crew  stations)  during  the  weapon  system  acquisition 
process.  The  approach  taken  is  to  identify  and  validate  HFE  tools  of 
various  kinds,  tools  that  DOD  HFE  practitioners  can  use  to  assist  them 
throughout  this  process.  What  tools  are  to  be  used,  when  to  use  them,  and 
how  to  use  the  outputs  must  be  specified. 

Assumptions  on  which  this  study  is  based  include  the  following. 

1.  Useful  tools  for  crew  station  analysis,  design,  and  evaluation  are 
needed  to  achieve  effective  crew  station  integration. 

2.  A  variety  of  potentially  useful  crew  station  analysis,  design,  and 
evaluation  tools  currently  exist  or  are  under  development. 

3.  Assessment  of  specific  tool  utility  and  validity  has  been  minimal  so 
far . 


During  the  first  phase  of  this  study,  a  detailed  literature  review 
was  carried  out  to  determine  (1)  what  previous  surveys  of  HFE  tools  had 
been  made  and  the  results  of  these  surveys,  (2)  what  things  should  be 
considered  in  setting  up  a  taxonomy  of  HFE  tools,  and  (3)  what  tool 
evaluation  and  validation  methods  have  been  proposed  and  used  for  other 
projects . 

A  HFE  Procedures  Taxonomy  was  developed,  based  on  (1)  the  five  phases 
of  the  weapon  system  acquisition  process  in  which  the  tools  might  be  used, 
and  (2)  the  115  HFE  procedures  or  tasks  identified  for  this  study  that  may 
be  carried  out  during  one  or  more  of  the  acquisition  phases  and  with  which 
the  tools  might  be  helpful.  Implemented  as  an  EXCEL  spreadsheet,  the 
taxonomy  includes  4  attributes  for  each  of  the  115  HFE  procedures;  (1)  the 
organizational  level  at  which  it  usually  is  applied,  (2)  resources 
currently  available  to  assist  with  the  procedure  usually  is  included,  and 
(4)  the  types  of  military  systems  for  which  the  procedure  is  applicable. 

A  HFE  Tools  Taxonomy  database  has  been  partially  implemented,  based 
on  the  Procedures  Taxonomy.  This  database  includes  12  attributes  for  each 
tool,  in  addition  to  the  4  attributes  of  the  procedure  to  which  the  tool 
applies . 

During  this  study  a  number  of  existing  or  nearly-complete  HFE  tools 
have  been  identified  as  potential  candidates  for  evaluation  and  validation 
during  the  next  phase  of  this  study.  These  tools  are  described  in  this 
report . 

Tool  evaluations  and  validations  are  critical  components  of  this 
project.  Tools  may  be  evaluated  to  determine  whether  they  appear  to  be 
useful.  This  is  done  through  subjective  analysis  based  on  written  reviews, 
on  conversations  with  persons  who  have  used  them,  and  on  personal 
experience,  to  determine  general  ease  of  use,  costs,  what  inputs  are 
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required,  what  outputs  are  provided,  etc.  This  is  the  process  being  used 
to  determine  which  tools  to  validate. 

Validation  is  necessary  to  determine  the  extent  to  which  the  tools 
actually  measure  or  do  what  was  intended.  Validation  requires  the  use  of 
testing  techniques  such  as  (1)  experimental  studies,  (2)  non-experimental 
systematic  comparisons,  and  (3)  systematic  violation  of  the  assumptions  on 
which  the  tool  is  based.  The  validation  process  that  will  be  used  to  test 
the  tools  identified  as  promising  candidates  is  described. 
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HFE  Procedures  Table 

The  following  Taxonomy  of  Human  Factors  Engineering 
Procedures  tables  are  taken  from  the  HFE  Tools  Taxonomy  data 
files.  The  tables  list  the  HFE  Procedures  that  may  be 
carried  out  during  the  military  system  acquisition  process. 
The  procedures  are  listed  in  the  tables  as  a  function  of  the 
system  acquisition  phase  or  phases  during  which  they  ?re  most 
likely  to  be  used,  and  in  the  approximate  order  in  which  they 
might  be  carried  out  during  that  acquisition  phase.  Not  all 
procedures  are  used  for  all  projects.  The  tables  are  part  of 
data  files  prepared  for  this  study  with  the  Microsoft  EXCEL 
spreadsheet  program. 

Notes  for  Table  Entries 

Task  No . 

These  numbers  represent  the  approximate  sequence  in 
which  the  HFE  Procedures  might  be  carried  out  in  a  normal 
(large-size)  project.  These  numbers  are  used  only  for 
reference  and  for  database  sorting. 

Acq.  Phs . 

The  military  system  acquisition  phase  or  phases  during 
which  the  HFE  Procedure  normally  is  applied  are  identified  as 
f ol lows . 

P  HFE  Procedure  carried  out  Pre-Milestone  0,  during 
the  Mission  Element  Needs  Statement  development 
and  mission  analysis  phase. 

0  HFE  Procedure  used  following  Milestone  0,  during 
the  concept  or  program  initiation  phase. 

1  HFE  Procedure  carried  out  following  Milestone  1, 
during  the  demonstration  and  validation  phase. 

2  HFE  Procedure  carried  out  following  Milestone  2, 
during  the  full-scale  engineering  development 
phase . 

3  HFE  Procedure  carried  out  following  Milestone  3, 
during  the  system  production  and  deployment  phase. 

Org.  Level 

Four  organization  levels  are  identified  at  which  the 
various  HFE  Procedures  (tasks)  usually  are  carried  out  (based 
on  Merriman,  1986)  : 

Acq  Major  milestones  and  events  associated  with  weapon 
system  acquisition  that  drive  all  other  tasks. 

These  tasks  generally  are  the  responsibility  of 
the  system  Program  Manager  (PMA) . 
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Mgt  SYSCOM-level  tasks  (NAVAIR,  NAVSEA,  etc.)  which 

drive  contractor  and  laboratory  efforts.  For  crew 
stations,  these  tasks  generally  are  the 
responsibility  of  AIR-5313. 

Cntr  Tasks  usually  (but  not  always)  carried  out  by  the 
contractor . 

Lab  Tasks  usually  (but  not  always)  carried  out  by  the 
Navy  support  laboratories. 

HFE  Procedure 

This  is  a  standard  procedure,  task,  or  technique  that 
may  be  used  at  some  point  during  the  acquisition  process  to 
improve  the  human  factors  engineering  of  the  system  being 
acquired . 

HFE  Resources 

Resources  are  existing  documents  or  other  aids  commonly 
used  to  carry  out  this  HFE  Procedure . 

Proj.  Size 

The  size  of  the  project  for  which  this  HFE  Procedure 
commonly  is  used  (most  procedures  are  not  cost-effective  for 
small  projects)  is  identified  as  follows. 

Lge  A  large  program,  such  as  a  new  aircraft  or  man-in- 
the-loop  missile. 

Med  A  medium-size  program,  such  as  a  major  aircraft 
modification  (procedures  identified  here  as 
appropriate  for  "Med”  projects  are  also  carried 
out  for  "Lge"  projects)  . 

Sml  A  small  project  such  as  a  minor  aircraft 

modification  (procedures  identified  here  as 
appropriate  for  "Sml"  projects  are  also  carried 
out  for  "Med"  and  "Lge"  projects) . 

Milit.  System 

The  kind  of  military  system  to  which  this  HFE  Procedure 
applies  is  identified  here:  shipboard,  subsurface,  aircraft, 
ground  vehicle,  weapon  system,  etc.  Most  of  the  procedures 
are  applicable  to  all  military  platforms  and  systems,  but  a 
few  are  identified  as  specific  to  aircraft  programs. 

No.  of  Tools 

Tools  are  systems  or  devices  that  can  be  used  to  assist 
a  human  factors  researcher  or  engineer  in  carrying  out  this 
HFE  Procedure.  The  quantity  of  existing  tools  that  have  been 
identified  to  date  for  this  procedure  are  noted  here. 
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Taxonomy  of  Human  Factors  Engineering  Procedures 


Task 

No. 

Acq. 

Phs. 

Org. 

Levei 

HFE  Procedure 

HFE  Resources 

Pro). 

Size 

Miiit. 

System 

No.  of 

Tools 

1 

P 

Acq 

MENS  Preparation 

Med 

All 

2 

P,0 

Mgt 

Define  HFE  Issues/Risks 

Med 

All 

3 

P,0 

Mqt 

Review  Similar  Projects 

Med 

All 

4 

P,0 

Mqt 

Cost/Benefit  Analysis 

Med 

All 

5 

P,0 

Lab 

Experimental  Studies 

SmI 

All 

6 

p,0 

Lab 

Data  Analysis 

SmI 

All 

Task 

No. 

Acq. 

Phs. 

Org. 

Levei 

HFE  Procedure 

HFE  Resources 

Pro). 

Size 

No.  of 
Tools 

7 

0 

Acq 

Concept  Studies 

Med 

All 

1 

8 

0 

Mqt 

Manpower  Constraints 

Med 

All 

9 

0 

Acq 

DemA/al  SOW 

MIL-STD-962 

Med 

All 

1 

1  0 

0 

Acq 

Evaluate  Dem/Val  Proposals 

Med 

All 

1  1 

0 

Acq 

System  Concept  Paper 

Med 

All 

1  2 

0 

-Mgt 

Plan  Project  Manaqement 

MIL-H-46855 

SmI 

All 

2 

13  0  Mat  Coordinate  Lab  Support  Requirem’ts  SmI  All  1 
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Taxonomy  of  Human  Factors  Engineering  Procedures 


Task 

Acq. 

Org. 

Proj. 

Mint. 

No.  of 

No. 

Phs. 

Level 

HFE  Procedure 

HFE  Resources  Size 

System 

Tools 

Prepare  Task  Aqreements 


Literature  Reviews 


Investiqate  Risk  Areas 


Concept  Investiqations 


FSDSOW 


Draft  HE  Program  Plan 


Review  HE  Program  Plan 


HE  Program  Review 


Mission  Analysis 


Mission  Analysis  Review 


Functional  Analysis 


Function  Allocation 


Performance  Requirements 


Manpower  Requirements 


Review  Manpower  Requirements 


Multicrew  Coordination  Requirem'ts 


Dynamic  Simulation  Plan 


Review  Dynamic  Simulation  Plan 


CSERIAC 


MIL-STD-962 


DI-HFAC-80740 


DI-HFAC-80740 


MIL-H-46855 


MIL-H-46855 


MIL-H-46855 


MIL-H-46855 


DI-HFAC-80742 


DI-HFAC-80742 


All 


All 


All 


All 


All 


All 


All 


All 


All 


All 


All 


All 


All 


All 


All 


Aircft. 


Aircft. 


!  1 

Cntr 

HE  Test  Plan 

DI-HFAC-80743 

Med 

All 

!  1 

Mqt 

Review  HE  Test  Plan 

Dl-HFAC-80743 

Med 

All 

34 

1 

Cntr 

Information  Analysis 

Med 

All 

1 

35 

1 

Cntr 

Equipment  Identification 

MIL-STD-1472 

Sml 

All 

36 

1 ,2 

Cntr 

Draft  HESAR 

DI-HFAC-80745 

Lge 

All 

37 

1 .2 

Mgt 

Review  HESAR 

Dl-HFAC-80745 

Lge 

All 

38 

1,2 

Cntr 

Gross  Task  Analysis 

HEL  TM  13-87 

Sml 

All 

39 

1 

Cntr 

Preliminary  Workstation  Layout 

MIL-STD-1472 

Sml 

All 

40 

1 

Cntr 

Crewstation  Mockups 

PMTC  TP-80-10 

Lge 

All 

4  1 

1 

Lab 

Crewstation  Review  Procedure 

Med 

All 

Crewstation  Design,  Review 


1st  Dem/Val  Design  Review 


1  1 ,2 

Cntr 

Design 

Displays/Controls 

MIL-STD-1472 

Sml 

All  ' 

1  1 ,2 

Mgt 

Review 

Displays/Controls 

MIL-STD-1472 

Sml 

All  ^ 

2nd  Dem/Val  Des.  Review 


Final  Dem/Val  Reports 


1 ,2 

Cntr 

Draft  HEDAD-0 

DI-HFAC-80746 

Med 

All 

1 .2 

Mgt 

Review  HEDAD-O 

DI-HFAC-80746 

Med 

All 

1 .2 

Cntr 

Critical  Task  Analysis 

DI-H-7055 

Med 

All 

• 

1,2 

Mgt 

Review  Critical  Task  Ana. 

DI-H-7055 

Med 

All 

1 

Mqt 

Maintenance  Mockup 

Med 

All 

1 

1,2 

Cntr 

Draft  HEDAD-M 

DI-HFAC-80747 

Lqe 

All 

- 

1 ,2 

Mgt 

Review  HEDAD-M 

DI-HFAC-80747 

Lge 

All 

1 

Cntr 

Maintainer  Critical  Task  Analysis 

DI-H-7055 

Lge 

All 

1 

Mgt 

Review  Maint.  Critical  Task 

Analysis  DI-H-7055 

Lge 

All 

1 .2 

Cntr 

HE  Progress  Report 

DI-HFAC-80741 

Med 

All 

_ L2 

Mqt 

Review  HE  Progress  Report 

DI-HFAC-80741 

Med 

All 

Aircft, 
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Taxonomy  of  Human  Factors  Engineering  Procedures 


Task 

No. 

Acq. 

Phs. 

Org. 

Level 

HFE  Procedure 

HFE  Resources 

Proj. 

Size 

Mint. 

System 

No.  of 
Tools 

59 

2 

Mql 

Design  Tradeoff  Studies 

Lqe 

All 

60 

2 

Mgt 

Resolve  Conflicting  Reqts. 

Med 

All 

61 

2 

Mgt 

Set  Up  ASAP  Team 

NAVAIRINST  5420.3: 

Lge 

Aircft. 

62 

2 

Cntr 

Job  Descriptions 

Med 

All 

63 

2 

Cntr 

Manpower  Estimates 

Med 

All 

5 

64 

2 

Mgt 

Review  Manpower  Est. 

Med 

All 

5 

65 

2 

Cntr 

Manpower/Training  Tradeoffs 

Med 

All 

66 

2 

Mgt 

Review  Manpower/Traininq  Trades 

Med 

All 

67 

2 

Cntr 

Time  line/link  analysis 

Med 

All 

68 

2 

Cntr 

Workload  Analysis 

MIL-H-46855 

SmI 

All 

9 

69 

2 

Cntr 

Anthropometry 

MIL-STD-1472 

Med 

All 

1  0 

70 

2 

Cntr 

System  Modeling 

MIL-STD-1472 

Lqe 

All 

71 

2 

Cntr 

Define  Operator-System  Interface 

MIL-STD-1472 

Med 

All 

1 

72 

2 

Mgt 

Review  Operator-System  Interface 

MIL-STD-1472 

Med 

All 

1 

73 

2 

Cntr 

Workstation  Layout 

MIL-STD-1472 

SmI 

All 

4 

74 

2 

Mgt 

Review  Workstation  Lay. 

MIL-STD-1472 

SmI 

All 

75 

2.3 

Cntr 

Conduct  Dynamic  Simulations 

Lqe 

Aircft. 

76 

2,3 

Mgt 

Review  Dynamic  Simulations 

Lqe 

Aircft. 

77 

2 

Cntr 

Define  Maintainer-System  Interface 

AD-141  0 

Med 

All 

2 

78 

2 

Mgt 

Review  Maintainer-System  Interface  AD-141 0 

Med 

All 

2 

79 

2,3 

Cntr 

Safety  Assessment 

SmI 

All 

80 

2,3 

Mgt 

Review  Safety  Assessment 

SmI 

All 

81 

2.3 

Cntr 

Health  Hazard  Assesment 

SmI 

All 

82 

2,3 

Mgt 

Rev.iew  Health  Hazard  Assessment 

SmI 

All 

83 

2 

Mc,t 

Preliminary  Design  Review 

Med 

All 

84 

2 

Mgt 

Critical  Design  Review 

Med 

All 

85 

2 

Cntr 

Human  Reliability  Prediction 

Lqe 

All 

86 

2 

Mgt 

Review  Human  Reliability  Prediction 

Lqe 

All 

87 

2 

Lab 

Lighting  Review  Procedure 

Med 

Aircft. 

88 

2 

Mgt 

Crewstation  Lighting  Review 

Med 

Aircft. 

89 

2 

Mgt 

Detail  Design  Review 

Med 

All 

90 

2 

Lab 

Night  Lighting  Review  Procedure 

Med 

Aircft. 

91 

2 

Mgt 

Night  Lighting  Review 

Med 

Aircft. 

92 

2 

Cntr 

System  Performance  Predictions 

Med 

All 

1 

93 

2 

Mgt 

Review  System  Performance  Predict. 

Med 

All 

1 

94 

2 

Mgt 

Developmental  Testing 

Med 

All 

95 

2 

Cntr 

HE  Test  Report 

DI-HFAC-80744 

Med 

All 

96 

2 

Mgt 

Review  HE  Test  Report 

DI-HFAC-80744 

Med 

All 

97 

2 

Cntr 

Operator  Training  Plan 

Med 

All 

1 

98 

2 

Mgt 

Review  Operator  Training  Plan 

Med 

All 

1 

99 

2 

Cntr 

Prepare  Users  Manuals 

Med 

All 

1 

1  00 

2 

Mgt 

Review  Users  Manuals 

Med 

All 

1 

101 

2 

Cntr 

Maintainer  Training  Plan 

Lge 

All 

1 

1  02 

2 

Mgt 

Review  Maintainer  Training  Plan 

Lqe 

All 

1 
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Taxonomy  of  Human  Factors  Engineering  Procedures 


Task 

No. 

Acq. 

Phs. 

Org, 

Level 

HFE  Procedure 

HFE  Resources 

Prol.  j 
Size  1 

Milit. 

System 

No.  of 
Tools 

3 

Acq 

TECHEVAL 

Med 

All 

3 

Acq 

OPEVAL 

Med 

All 

3 

Mqt 

Document  T&E  Deficiencies 

Med 

All 

106 

3 

Lab 

T&E  Deficiency  Corrections 

Med 

All 

1  07 

3 

Mgt 

Define  Product  Improvements 

Med 

All 

108 

3 

Lab 

Functional  Configuration  Audit 

Lqe 

All 

1  09 

3 

Lab 

Physical  Configuration  Audit 

Lge 

All 

1  1  0 

3 

Lab 

Analyze  Hardware  Trouble  Reports 

Med 

All 

1 1 1 

3 

Lab 

Analyze  Software  Trouble  Reports 

Med 

All 

112 

3 

Mqt 

Prepare  Enq.  Change  Proposals 

MIL-STD-480 

Med 

Ail 

113 

3 

Lab 

Review  Enq.  Change  Proposals 

MIL-STD-480 

Med 

All 

1 1  4 

3 

Acq 

Production  Readiness  Review 

Med 

All 

115 

3 

Acq 

Production  SOW 

MIL-STD-962 

Med 

All 
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TOOL  TYPES  AND  THE  HFE  TASKS  THAT  THEY  SUPPORT 

1.  HFE  Principles,  Standards,  Specifications 

a.  Apply  HFE  criteria  &  principles 

2.  HFE  Data  Base 

a.  Review/ summarize  the  results  of  previous  research 

3.  HFE  Acquisition  Support 

a.  Program  planning 

b.  Specify  design  requirements 

c.  Develop  work  statements 

d.  Develop  data  requirements 

e.  Develop  test  requirements 

4.  System  Analysis 

a.  Functional  analysis 

b.  Function  allocation 

c.  Comparability  analysis 

5.  Task  Analysis 

a.  Analyze  tasks 

b.  Network  analysis 

6.  Workload  Assessment 

a.  Assess  mental  workload 

b.  Assess  physical  workload 

7.  Crewstation  Layout 

a.  Design  workspace  environment  (habitability) 

b.  Design  crewstation  parameters 

1)  internal  &  external  visibility 

2)  lighting 

3)  control /display  arrangement 
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4)  accessibility 

5)  life  support  features 

6)  escape  features 

c.  Prepare  design  mockups 

8.  Man-Modeling/Anthropometrics 

a.  Crewstation  geometry 

b.  Anthropometric  matrix  preparation  (design/test  cases) 

c.  Crew  accommodation  assessment  (crewstation  mapping) 

d.  Aircrew  life  support  systems  design/user  interface  (e.g.  oxygen 
mask) 

e.  Subsystem  design/ integration  (e.g.  ejection  seat) 

9.  Hardware  Design/Human-Equipment  Interface  (Operability  and 
Maintainabi 1 ity) 

a.  Prepare  specifications  for  hardware  components 

b.  Develop  mockups 

c.  Design  of  enclosure  for  hardware  components 

1)  weight,  size 

2)  layout  of  external  features  (control  panel,  indicators, 
handles) 

3)  layout  of  internal  components  (remove/replace  assemblies) 

d.  Rack  design  (location  of  equipment,  accessibility) 

e.  Input  and  control  device  design 

f.  Display  device  design 

g.  Life  support  equipment 

10.  Software  Design/Human-Computer  Interface  (Formats,  Logic) 

a.  Prepare  specifications  for  software 

b.  Develop  prototype  formats 

c.  Link  formats  to  assess  logical  flow 
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11.  Test  &  Evaluation 

a.  Verify  design  conformance  to  specifications  (design  criteria 
checklist).  Evaluation  categories  include: 

1)  Design  (e.g.,  workspace  and  anthropometries) 

2)  Performance  (e.g.,  human  error  evaluation) 

3)  Maintainability  (e.g.,  accessibility) 

4)  Habitability  (e.g.,  noise  measurements) 

b.  Demonstrate  conformance  to  system,  equipment,  and  facility  design 
criteria  (specifications,  drawings) 

c.  Confirm  compliance  with  performance  requirements  where  the 
operator  or  maintainer  is  a  significant  part  of  such  system 
performance 

d.  Obtain  quantitative  measures  of  system  performance  which  are  a 
function  of  man-machine  interaction 

e.  Determine  if  undesirable  design  or  procedural  aspects  have  been 
introduced 
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